You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/11/06 16:04:28 UTC
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3722 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3722/
1 tests failed.
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9839 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001/init-core-data-001
[junit4] 2> 48955 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
[junit4] 2> 48956 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 48956 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 48962 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDoubleAssignment
[junit4] 2> 48963 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 49063 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:51889
[junit4] 2> 49063 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49064 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49068 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68781cb5 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49068 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49069 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49069 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49070 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49072 INFO (zkCallback-49-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a44db2a name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49073 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49073 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49076 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49077 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49079 INFO (zkCallback-50-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6759c069 name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 49085 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 49088 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 49090 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 49092 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 49094 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 49095 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 49099 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49099 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49101 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76ac9512 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49101 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49102 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49105 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 49108 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 49109 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 49109 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 49110 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49111 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49113 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@190452f0 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49113 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49113 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49114 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 49121 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 49123 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 49125 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801242931204-127.0.0.1:51889_solr-n_0000000000
[junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51889_solr
[junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 49128 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) starting
[junit4] 2> 49130 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 49133 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 49136 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 49139 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 49141 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 49144 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 49150 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 49150 INFO (OverseerCollectionConfigSetProcessor-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 49151 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 49152 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 49157 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 49161 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49161 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49161 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49654 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 49656 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 49656 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 49656 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 49661 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801242931203-node1_core1-n_0000000000
[junit4] 2> 49662 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 49665 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 49668 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 49669 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"}
[junit4] 2> 49673 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49673 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49678 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49678 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 49679 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 49680 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 49685 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c4f31de name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 49686 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 49686 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 49695 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 49698 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 49700 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 49700 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 49700 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 49703 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 49704 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 49705 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 49705 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 49708 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801242931205-node1_core1-n_0000000001
[junit4] 2> 49710 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 49714 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 49715 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"}
[junit4] 2> 49722 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 49831 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49831 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49831 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49836 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49836 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 49845 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51889 51889
[junit4] 2> 50045 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51889 51889
[junit4] 2> 50046 WARN (Thread-85) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/collections
[junit4] 2> 4 /solr/live_nodes
[junit4] 2>
[junit4] 2> 50047 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDoubleAssignment
[junit4] 2> 50047 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) closing
[junit4] 2> 50047 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51889_solr
[junit4] 2> 50057 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 50060 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
[junit4] 2> 50061 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 50161 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:56334
[junit4] 2> 50161 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 50162 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 50165 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a4111b2 name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 50165 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 50165 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 50168 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 50168 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 50170 INFO (zkCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51182c3c name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 50174 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 50174 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 50180 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f355f55 name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 50180 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 50181 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 50181 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 50183 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 50185 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 50187 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 50189 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 50191 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 50193 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 50194 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 50196 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5336a302 name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 50196 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 50197 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 50200 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 50203 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 50205 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 50205 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 50205 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 50208 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 50210 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 50211 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 50212 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801314693122-127.0.0.1:56334_solr-n_0000000000
[junit4] 2> 50213 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56334_solr
[junit4] 2> 50213 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 50215 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) starting
[junit4] 2> 50217 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 50220 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 50224 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 50227 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 50230 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 50233 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 50238 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 50238 INFO (OverseerCollectionConfigSetProcessor-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 50240 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 50246 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 50246 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 50741 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 50743 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 50743 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 50749 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801314693123-node1_core1-n_0000000000
[junit4] 2> 50753 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 50757 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 50761 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 50761 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"}
[junit4] 2> 50764 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 50764 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 50764 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 50785 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56334 56334
[junit4] 2> 50887 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56334 56334
[junit4] 2> 50889 WARN (Thread-86) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/collections
[junit4] 2> 2 /solr/live_nodes
[junit4] 2>
[junit4] 2> 50889 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
[junit4] 2> 50889 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) closing
[junit4] 2> 50890 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56334_solr
[junit4] 2> 50897 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset -Dtests.seed=7B352AEEA690CBAB -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hu -Dtests.timezone=NET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.85s J2 | OverseerTest.testOverseerStatsReset <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected:<1> but was:<2>
[junit4] > at __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 50910 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem
[junit4] 2> 50911 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 51011 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:60811
[junit4] 2> 51011 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 51012 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51015 INFO (zkCallback-60-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f19b154 name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 51015 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51016 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 51018 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 51018 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51020 INFO (zkCallback-61-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22e9d784 name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 51020 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51021 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 51021 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 51024 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 51024 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51026 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f0d6162 name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 51026 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51027 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 51027 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 51029 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 51035 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 51037 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 51039 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 51041 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 51041 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51043 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d0bfa0d name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 51043 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51044 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 51044 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 51046 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 51048 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 51050 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 51051 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801370398723-127.0.0.1:60811_solr-n_0000000000
[junit4] 2> 51051 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60811_solr
[junit4] 2> 51052 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 51053 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) starting
[junit4] 2> 51055 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 51058 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 51060 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 51063 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 51065 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 51068 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 51072 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 51073 INFO (OverseerCollectionConfigSetProcessor-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 51073 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 51074 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 51075 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 51076 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 51077 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51081 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@286bb11a name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 51081 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51081 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 51083 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 51086 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
[junit4] 2> 51090 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 51090 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 51091 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 51095 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 0
[junit4] 2> 51095 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 51096 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 51096 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 51098 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 51098 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 51098 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 51594 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 51596 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 51596 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 51597 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 51603 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
[junit4] 2> 51604 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
[junit4] 2> 51611 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 51615 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 51618 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 1
[junit4] 2> 51618 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 51619 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 51619 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 51722 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 51722 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 51723 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52117 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 52122 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
[junit4] 2> 52123 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
[junit4] 2> 52126 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 52130 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core2"} current state version: 2
[junit4] 2> 52133 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 2
[junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 52238 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52238 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52238 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52631 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 52638 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
[junit4] 2> 52639 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 52642 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 52646 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core3"} current state version: 3
[junit4] 2> 52750 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52750 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52750 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52799 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 4
[junit4] 2> 52800 ERROR (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 52802 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""} current state version: 4
[junit4] 2> 52802 ERROR (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}
[junit4] 2> 52804 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 4
[junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection2
[junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 52909 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52909 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 52909 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53302 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard3/election
[junit4] 2> 53304 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 53304 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 53305 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 53309 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
[junit4] 2> 53310 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard3
[junit4] 2> 53313 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 53317 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core1"} current state version: 5
[junit4] 2> 53320 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 5
[junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 53426 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53426 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53426 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53819 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard2/election
[junit4] 2> 53826 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
[junit4] 2> 53827 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard2
[junit4] 2> 53830 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 53833 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core2"} current state version: 6
[junit4] 2> 53836 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 6
[junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 53941 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53941 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 53941 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 54335 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard1/election
[junit4] 2> 54345 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
[junit4] 2> 54347 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard1
[junit4] 2> 54350 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 54355 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core3"} current state version: 7
[junit4] 2> 54460 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 54460 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 54460 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 54506 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 54507 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 56007 WARN (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 56010 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60811 60811
[junit4] 2> 56253 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60811 60811
[junit4] 2> 56256 WARN (Thread-87) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/live_nodes
[junit4] 2>
[junit4] 2> 56256 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
[junit4] 2> 56256 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) closing
[junit4] 2> 56256 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60811_solr
[junit4] 2> 56259 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 56267 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplay
[junit4] 2> 56268 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 56368 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:43766
[junit4] 2> 56368 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 56371 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 56376 INFO (zkCallback-66-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@de91d1d name:ZooKeeperConnection Watcher:127.0.0.1:43766/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 56376 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 56376 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 56377 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 56377 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 56379 INFO (zkCallback-67-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20a01f5b name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 56379 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 56380 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 56381 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 56382 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 56383 INFO (zkCallback-68-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6318cd44 name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 56383 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 56384 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 56384 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 56387 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 56389 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 56391 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 56393 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 56395 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 56397 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 56401 INFO (TEST-OverseerTest.testReplay-seed
[...truncated too long message...]
nit4] 2> 105808 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 105810 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 105813 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 105815 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 105817 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 105819 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 105822 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 105829 INFO (OverseerCollectionConfigSetProcessor-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 105833 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 105834 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 105837 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"} current state version: 0
[junit4] 2> 105837 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 105838 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 105840 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 105840 INFO (zkCallback-154-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 105896 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator node=core_node1 is already registered
[junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 106000 INFO (zkCallback-154-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 106000 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 106053 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40794 40794
[junit4] 2> 106225 INFO (Thread-96) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40794 40794
[junit4] 2> 106226 WARN (Thread-96) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/collections
[junit4] 2> 2 /solr/live_nodes
[junit4] 2>
[junit4] 2> 106226 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateChange
[junit4] 2> 106226 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818804959215619-127.0.0.1:40794_solr-n_0000000000) closing
[junit4] 2> 106227 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40794_solr
[junit4] 2> 106228 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 109228 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=hu, timezone=NET
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=198038552,total=343932928
[junit4] 2> NOTE: All tests run in this JVM: [HardAutoCommitTest, TestPerFieldSimilarity, FieldMutatingUpdateProcessorTest, VMParamsZkACLAndCredentialsProvidersTest, VersionInfoTest, LukeRequestHandlerTest, DeleteLastCustomShardedReplicaTest, ActionThrottleTest, OverseerTest]
[junit4] Completed [24/545] on J2 in 60.31s, 12 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 1651 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2156 tests, 1 failure, 51 ignored (37 assumptions) [seed: 7B352AEEA690CBAB]
Total time: 60 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3725 - Still Failing
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
https://issues.apache.org/jira/browse/SOLR-8249 should fix this. Sorry for
the noise.
On Sat, Nov 7, 2015 at 1:49 PM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3725/
>
> 1 tests failed.
> FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
>
> Error Message:
> expected:<1> but was:<2>
>
> Stack Trace:
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> __randomizedtesting.SeedInfo.seed([B47F6E4FF608FD26:1F2B897363D25E28]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:472)
> at org.junit.Assert.assertEquals(Assert.java:456)
> at
> org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 10417 lines...]
> [junit4] Suite: org.apache.solr.cloud.OverseerTest
> [junit4] 2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_B47F6E4FF608FD26-001/init-core-data-001
> [junit4] 2> 960760 INFO
> (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
> [junit4] 2> 960760 INFO
> (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ####initCore
> [junit4] 2> 960760 INFO
> (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ####initCore end
> [junit4] 2> 960800 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testReplay
> [junit4] 2> 960800 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 960811 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 960811 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 960911 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:54238
> [junit4] 2> 960911 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 960937 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 960952 INFO (zkCallback-535-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7a62b1
> name:ZooKeeperConnection Watcher:127.0.0.1:54238/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 960955 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 960955 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 960955 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 960964 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 960969 INFO (zkCallback-536-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@2911d84
> name:ZooKeeperConnection Watcher:127.0.0.1:54238 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 960969 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 960969 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 960976 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 960982 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961017 INFO (zkCallback-537-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7e85766f
> name:ZooKeeperConnection Watcher:127.0.0.1:54238 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961017 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961017 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961017 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> [junit4] 2> 961020 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> [junit4] 2> 961021 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> [junit4] 2> 961022 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> [junit4] 2> 961023 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> [junit4] 2> 961024 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> [junit4] 2> 961025 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 961027 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> [junit4] 2> 961029 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961042 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961055 INFO (zkCallback-538-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@44977f5d
> name:ZooKeeperConnection Watcher:127.0.0.1:54238/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961056 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961056 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961056 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> [junit4] 2> 961057 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> [junit4] 2> 961059 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> [junit4] 2> 961060 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 961060 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94822931338756099-127.0.0.1:54238_solr-n_0000000000
> [junit4] 2> 961061 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54238
> _solr
> [junit4] 2> 961061 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> [junit4] 2> 961062 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931338756099-127.0.0.1:54238_solr-n_0000000000) starting
> [junit4] 2> 961063 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> [junit4] 2> 961065 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> [junit4] 2> 961067 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> [junit4] 2> 961068 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> [junit4] 2> 961070 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> [junit4] 2> 961073 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> [junit4] 2> 961089 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> [junit4] 2> 961090 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 961092 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 961092 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 961094 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core2",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 961094 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core2",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 961105 INFO
> (OverseerCollectionConfigSetProcessor-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> [junit4] 2> 961115 INFO (zkCallback-538-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 961122 INFO (zkCallback-535-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 961150 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core3",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"} current state version: 2
> [junit4] 2> 961150 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "shard":"s1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core3",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 961253 INFO (zkCallback-535-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 961253 INFO (zkCallback-538-thread-3) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 961319 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:54238 54238
> [junit4] 2> 961441 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:54238 54238
> [junit4] 2> 961442 WARN (Thread-1963) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> [junit4] 2> Maximum concurrent create/delete watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/aliases.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent data watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/clusterstate.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent children watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/collections
> [junit4] 2> 2 /solr/live_nodes
> [junit4] 2>
> [junit4] 2> 961442 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testReplay
> [junit4] 2> 961442 INFO
> (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931338756099-127.0.0.1:54238_solr-n_0000000000) closing
> [junit4] 2> 961445 INFO
> (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54238_solr
> [junit4] 2> 961469 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testOverseerFailure
> [junit4] 2> 961469 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 961481 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
> could not read the data
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> [junit4] 2> 961488 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 961488 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 961581 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:44090
> [junit4] 2> 961581 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961597 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961613 INFO (zkCallback-540-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7306b0a
> name:ZooKeeperConnection Watcher:127.0.0.1:44090 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961613 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961613 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961621 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961641 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961641 INFO (zkCallback-541-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@14c7ad60
> name:ZooKeeperConnection Watcher:127.0.0.1:44090 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961641 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961641 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961641 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> [junit4] 2> 961667 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961674 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961675 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@2a828b25
> name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961675 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961675 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961675 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> [junit4] 2> 961677 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> [junit4] 2> 961678 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> [junit4] 2> 961679 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> [junit4] 2> 961680 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> [junit4] 2> 961681 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 961682 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961718 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961745 INFO (zkCallback-543-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@70bf6912
> name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961745 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961745 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961747 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 961748 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
> [junit4] 2> 961749 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> [junit4] 2> 961750 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 961767 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> [junit4] 2> 961775 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 961798 INFO (zkCallback-544-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@471a9a7a
> name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 961798 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 961799 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 961799 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> [junit4] 2> 961800 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> [junit4] 2> 961801 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> [junit4] 2> 961802 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 961803 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94822931383648260-127.0.0.1:44090_solr-n_0000000000
> [junit4] 2> 961803 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44090
> _solr
> [junit4] 2> 961803 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> [junit4] 2> 961804 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) starting
> [junit4] 2> 961805 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> [junit4] 2> 961807 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> [junit4] 2> 961808 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> [junit4] 2> 961810 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> [junit4] 2> 961811 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> [junit4] 2> 961813 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> [junit4] 2> 961816 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> [junit4] 2> 961825 INFO
> (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> [junit4] 2> 961826 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> [junit4] 2> 961826 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 962843 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"recovering",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 0
> [junit4] 2> 962844 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"recovering",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"}
> [junit4] 2> 962844 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
> [junit4] 2> 962844 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> [junit4] 2> 962845 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 962845 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 962845 INFO (zkCallback-544-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 963343 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 963344 INFO (zkCallback-544-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 963345 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 963345 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 963349 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
> [junit4] 2> 963350 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
> [junit4] 2> 963353 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> [junit4] 2> 963356 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://node1/solr/",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "core":"core1"} current state version: 1
> [junit4] 2> 963357 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://node1/solr/",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "core":"core1"}
> [junit4] 2> 963360 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"leader",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://node1/solr/",
> [junit4] 2> "core":"core1"} current state version: 1
> [junit4] 2> 963361 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 963363 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 1
> [junit4] 2> 963363 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"}
> [junit4] 2> 963363 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
> [junit4] 2> 963366 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000001
> [junit4] 2> 963366 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector was going to be leader
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000001
> , seq(0)
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
> [junit4] 2> 963466 INFO (zkCallback-544-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 963466 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 963468 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 963828 INFO
> (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:355)
> [junit4] 2> at
> org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:172)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> 963829 INFO
> (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor According to ZK I
> (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) am no longer a
> leader.
> [junit4] 2> 964522 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 964524 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000002
> [junit4] 2> 964524 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector was going to be leader
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000002
> , seq(0)
> /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
> [junit4] 2> 964525 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 964525 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 964529 INFO (zkCallback-546-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6254332a
> name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 964529 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 964529 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 964529 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) closing
> [junit4] 2> 964530 INFO
> (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44090_solr
> [junit4] 2> 964530 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> [junit4] 2> 964531 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
> could not read the data
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> [junit4] 2> 964532 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 964533 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94822931383648261-127.0.0.1:44090_solr-n_0000000001
> [junit4] 2> 964533 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44090
> _solr
> [junit4] 2> 964533 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> [junit4] 2> 964535 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931383648261-127.0.0.1:44090_solr-n_0000000001) starting
> [junit4] 2> 964542 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> [junit4] 2> 964543 INFO
> (OverseerCollectionConfigSetProcessor-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> [junit4] 2> 964544 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> [junit4] 2> 964546 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 964553 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"recovering",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 2
> [junit4] 2> 964553 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"recovering",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"1",
> [junit4] 2> "base_url":"http://node1/solr/"}
> [junit4] 2> 964554 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
> [junit4] 2> 964556 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 964556 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 964556 INFO (zkCallback-546-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 964608 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"deletecore",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"core_node1",
> [junit4] 2> "collection":"collection1"} current state version: 3
> [junit4] 2> 964710 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.o.ZkStateWriter going to delete_collection
> /collections/collection1/state.json
> [junit4] 2> 964716 INFO (zkCallback-546-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 964716 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 964716 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 964717 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 964717 INFO (zkCallback-546-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 964717 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 965218 INFO (zkCallback-546-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> [junit4] 2> 965218 INFO (zkCallback-542-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> [junit4] 2> 965218 INFO (zkCallback-543-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> [junit4] 2> 965223 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:44090 44090
> [junit4] 2> 965408 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:44090 44090
> [junit4] 2> 965408 WARN (Thread-1964) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> [junit4] 2> Maximum concurrent create/delete watches above limit:
> [junit4] 2>
> [junit4] 2> 4 /solr/aliases.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent data watches above limit:
> [junit4] 2>
> [junit4] 2> 4 /solr/clusterstate.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent children watches above limit:
> [junit4] 2>
> [junit4] 2> 4 /solr/collections
> [junit4] 2> 4 /solr/live_nodes
> [junit4] 2> 2 /solr/overseer/queue
> [junit4] 2> 2 /solr/overseer/collection-queue-work
> [junit4] 2> 2 /solr/overseer/queue-work
> [junit4] 2>
> [junit4] 2> 965409 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testOverseerFailure
> [junit4] 2> 965409 INFO
> (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931383648261-127.0.0.1:44090_solr-n_0000000001) closing
> [junit4] 2> 965409 INFO
> (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44090_solr
> [junit4] 2> 965410 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
> could not read the data
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> [junit4] 2> 965415 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testStateChange
> [junit4] 2> 965416 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 965418 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 965418 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 965518 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:50085
> [junit4] 2> 965518 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 965519 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 965527 INFO (zkCallback-548-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6e204284
> name:ZooKeeperConnection Watcher:127.0.0.1:50085/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 965527 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 965527 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 965527 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 965529 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 965533 INFO (zkCallback-549-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@2bc872ab
> name:ZooKeeperConnection Watcher:127.0.0.1:50085 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 965533 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 965534 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 965534 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [
> ] o.a.z.s.NIOServerCnxn caught end of stream exception
> [junit4] 2> EndOfStreamException: Unable to read additional data from
> client sessionid 0x150e0f435010001, likely client has closed socket
> [junit4] 2> at
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> [junit4] 2> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4] 2> at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> 965535 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 965535 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 965553 INFO (zkCallback-550-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@2eb503e2
> name:ZooKeeperConnection Watcher:127.0.0.1:50085 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 965553 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 965553 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 965553 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> [junit4] 2> 965560 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> [junit4] 2> 965561 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> [junit4] 2> 965562 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> [junit4] 2> 965564 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> [junit4] 2> 965565 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> [junit4] 2> 965566 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 965567 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 965577 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 965577 INFO (zkCallback-551-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@2bfbf3f8
> name:ZooKeeperConnection Watcher:127.0.0.1:50085/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 965577 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 965578 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 965578 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> [junit4] 2> 965579 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> [junit4] 2> 965580 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> [junit4] 2> 965582 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 965583 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94822931640680451-127.0.0.1:50085_solr-n_0000000000
> [junit4] 2> 965583 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50085
> _solr
> [junit4] 2> 965583 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> [junit4] 2> 965584 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931640680451-127.0.0.1:50085_solr-n_0000000000) starting
> [junit4] 2> 965586 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> [junit4] 2> 965588 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> [junit4] 2> 965590 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> [junit4] 2> 965592 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> [junit4] 2> 965594 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> [junit4] 2> 965596 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> [junit4] 2> 965599 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> [junit4] 2> 965600 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> [junit4] 2> 965600 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 965605 INFO
> (OverseerCollectionConfigSetProcessor-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> [junit4] 2> 965607 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"} current state version: 0
> [junit4] 2> 965608 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"recovering"}
> [junit4] 2> 965608 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> [junit4] 2> 965609 INFO (zkCallback-548-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 965609 INFO (zkCallback-551-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 965660 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"active"} current state version: 1
> [junit4] 2> 965660 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "base_url":"http://127.0.0.1/solr",
> [junit4] 2> "node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "roles":"",
> [junit4] 2> "state":"active"}
> [junit4] 2> 965660 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator node=core_node1 is already registered
> [junit4] 2> 965660 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
> [junit4] 2> 965763 INFO (zkCallback-548-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 965763 INFO (zkCallback-551-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> [junit4] 2> 965812 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:50085 50085
> [junit4] 2> 965969 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:50085 50085
> [junit4] 2> 965970 WARN (Thread-1965) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> [junit4] 2> Maximum concurrent create/delete watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/aliases.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent data watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/clusterstate.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent children watches above limit:
> [junit4] 2>
> [junit4] 2> 2 /solr/collections
> [junit4] 2> 2 /solr/live_nodes
> [junit4] 2>
> [junit4] 2> 965970 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testStateChange
> [junit4] 2> 965970 INFO
> (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931640680451-127.0.0.1:50085_solr-n_0000000000) closing
> [junit4] 2> 965970 INFO
> (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:50085_solr
> [junit4] 2> 965977 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer
> could not read the data
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> [junit4] 2> 965982 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testShardAssignment
> [junit4] 2> 965983 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 965983 INFO (Thread-1966) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 965983 INFO (Thread-1966) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 966083 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:37594
> [junit4] 2> 966083 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 966089 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 966094 INFO (zkCallback-553-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@3edc922
> name:ZooKeeperConnection Watcher:127.0.0.1:37594 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 966096 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 966096 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 966098 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 966101 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 966113 INFO (zkCallback-554-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6cc287e
> name:ZooKeeperConnection Watcher:127.0.0.1:37594 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 966115 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 966115 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 966115 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> [junit4] 2> 966122 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 966123 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 966137 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@29d2100c
> name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 966137 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 966137 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 966138 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> [junit4] 2> 966140 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> [junit4] 2> 966141 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> [junit4] 2> 966142 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> [junit4] 2> 966145 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> [junit4] 2> 966147 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 966148 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 966151 INFO (zkCallback-556-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@131dfc89
> name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 966151 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 966151 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 966152 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> [junit4] 2> 966153 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> [junit4] 2> 966154 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> [junit4] 2> 966155 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 966156 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94822931677773827-127.0.0.1:37594_solr-n_0000000000
> [junit4] 2> 966156 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37594
> _solr
> [junit4] 2> 966156 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> [junit4] 2> 966159 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822931677773827-127.0.0.1:37594_solr-n_0000000000) starting
> [junit4] 2> 966160 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> [junit4] 2> 966162 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> [junit4] 2> 966164 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> [junit4] 2> 966167 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> [junit4] 2> 966170 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> [junit4] 2> 966172 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> [junit4] 2> 966176 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> [junit4] 2> 966177 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> [junit4] 2> 966178 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 966183 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 966184 INFO
> (OverseerCollectionConfigSetProcessor-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> [junit4] 2> 966187 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> [junit4] 2> 966193 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> [junit4] 2> 966197 INFO (zkCallback-558-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@22fcbcf5
> name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4] 2> 966197 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> [junit4] 2> 966197 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> [junit4] 2> 966199 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> [junit4] 2> 966202 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
> [junit4] 2> 966203 INFO (zkCallback-556-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> [junit4] 2> 966206 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> [junit4] 2> 966206 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 0
> [junit4] 2> 966208 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core1",
> [junit4] 2> "core_node_name":"node1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> [junit4] 2> 966208 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
> [junit4] 2> 966208 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
> [junit4] 2> 966211 INFO (zkCallback-556-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 966211 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 966214 INFO (zkCallback-558-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> [junit4] 2> 966217 INFO (zkCallback-558-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 966706 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard2/election
> [junit4] 2> 966707 INFO (zkCallback-558-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 966707 INFO (zkCallback-556-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 966707 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> [junit4] 2> 966710 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard2/election/94822931677773828-127.0.0.1_core1-n_0000000000
> [junit4] 2> 966711 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
> [junit4] 2> 966713 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> [junit4] 2> 966715 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"leader",
> [junit4] 2> "shard":"shard2",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> [junit4] 2> "core":"core1"} current state version: 1
> [junit4] 2> 966717 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core2",
> [junit4] 2> "core_node_name":"node2",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 1
> [junit4] 2> 966717 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core2",
> [junit4] 2> "core_node_name":"node2",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> [junit4] 2> 966717 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> [junit4] 2> 966717 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> [junit4] 2> 966821 INFO (zkCallback-556-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 966821 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 966821 INFO (zkCallback-558-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 967216 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> [junit4] 2> 967220 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94822931677773828-127.0.0.1_core2-n_0000000000
> [junit4] 2> 967220 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
> [junit4] 2> 967222 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> [junit4] 2> 967224 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"leader",
> [junit4] 2> "shard":"shard1",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> [junit4] 2> "core":"core2"} current state version: 2
> [junit4] 2> 967226 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core3",
> [junit4] 2> "core_node_name":"node3",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 2
> [junit4] 2> 967227 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core3",
> [junit4] 2> "core_node_name":"node3",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> [junit4] 2> 967227 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> [junit4] 2> 967227 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
> [junit4] 2> 967329 INFO (zkCallback-558-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 967329 INFO (zkCallback-556-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 967329 INFO (zkCallback-555-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 967726 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard3/election
> [junit4] 2> 967730 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard3/election/94822931677773828-127.0.0.1_core3-n_0000000000
> [junit4] 2> 967731 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
> [junit4] 2> 967733 INFO
> (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> [junit4] 2> 967736 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"leader",
> [junit4] 2> "shard":"shard3",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> [junit4] 2> "core":"core3"} current state version: 3
> [junit4] 2> 967738 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core4",
> [junit4] 2> "core_node_name":"node4",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 3
> [junit4] 2> 967739 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> [junit4] 2> "operation":"state",
> [junit4] 2> "state":"active",
> [junit4] 2> "node_name":"127.0.0.1",
> [junit4] 2> "core":"core4",
> [junit4] 2> "core_node_name":"node4",
> [junit4] 2> "collection":"collection1",
> [junit4] 2> "numShards":"3",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> [junit4] 2> 967739 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> [junit4] 2> 967739 INFO
> (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
> [junit4] 2> 967841 INFO (zkCallback-556-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - up
>
> [...truncated too long message...]
>
> Changed path:/clusterstate.json, has occurred - updating... (live nodes
> size: 1)
> [junit4] 2> 1009218 INFO (zkCallback-656-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 1009218 INFO (zkCallback-657-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 1009614 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection2/leader_elect/shard3/election
> [junit4] 2> 1009617 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection2/leader_elect/shard3/election/94822934316384260-127.0.0.1_core3-n_0000000000
> [junit4] 2> 1009618 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard3
> [junit4] 2> 1009619 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> [junit4] 2> 1009621 INFO
> (OverseerStateUpdate-94822934316384259-127.0.0.1:44386_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> [junit4] 2> "operation":"leader",
> [junit4] 2> "shard":"shard3",
> [junit4] 2> "collection":"collection2",
> [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> [junit4] 2> "core":"core3"} current state version: 7
> [junit4] 2> 1009724 INFO (zkCallback-656-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 1009724 INFO (zkCallback-657-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 1009724 INFO (zkCallback-659-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> [junit4] 2> 1009773 INFO (zkCallback-657-thread-2) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> [junit4] 2> 1009773 INFO (zkCallback-659-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> [junit4] 2> 1011273 WARN (zkCallback-659-thread-1) [ ]
> o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
> ZK
> [junit4] 2> 1011275 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:44386 44386
> [junit4] 2> 1011433 INFO (Thread-1974) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:44386 44386
> [junit4] 2> 1011434 WARN (Thread-1974) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> [junit4] 2> Maximum concurrent create/delete watches above limit:
> [junit4] 2>
> [junit4] 2> 3 /solr/aliases.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent data watches above limit:
> [junit4] 2>
> [junit4] 2> 3 /solr/clusterstate.json
> [junit4] 2>
> [junit4] 2> Maximum concurrent children watches above limit:
> [junit4] 2>
> [junit4] 2> 3 /solr/collections
> [junit4] 2> 3 /solr/live_nodes
> [junit4] 2>
> [junit4] 2> 1011434 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
> [junit4] 2> 1011434 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94822934316384259-127.0.0.1:44386_solr-n_0000000000) closing
> [junit4] 2> 1011435 INFO
> (OverseerStateUpdate-94822934316384259-127.0.0.1:44386_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44386_solr
> [junit4] 2> 1011436 ERROR (OverseerExitThread) [ ]
> o.a.s.c.Overseer could not read the data
> [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> [junit4] 2> 1014435 INFO
> (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ###deleteCore
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_B47F6E4FF608FD26-001
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {},
> docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {},
> locale=pt_PT, timezone=America/Port-au-Prince
> [junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_80 (64-bit)/cpus=4,threads=1,free=287501920,total=506462208
> [junit4] 2> NOTE: All tests run in this JVM: [RequestHandlersTest,
> TestReloadAndDeleteDocs, DateRangeFieldTest, TestWriterPerf,
> RollingRestartTest, LukeRequestHandlerTest, QueryParsingTest,
> TestGroupingSearch, PrimitiveFieldTypeTest, TestRestManager,
> StatsComponentTest, TestPerFieldSimilarity, TestSolrIndexConfig,
> HttpPartitionTest, TestConfigSetImmutable, BadComponentTest, SampleTest,
> CoreAdminHandlerTest, TestFileDictionaryLookup, TestStressReorder,
> AddSchemaFieldsUpdateProcessorFactoryTest, PluginInfoTest,
> TestCollationField, SolrCoreCheckLockOnStartupTest,
> XmlUpdateRequestHandlerTest, TestReversedWildcardFilterFactory,
> TestSchemaVersionResource, TestOverriddenPrefixQueryForCustomFieldType,
> TestSolrCloudWithKerberosAlt, TestRequestStatusCollectionAPI,
> QueryResultKeyTest, BinaryUpdateRequestHandlerTest, UpdateParamsTest,
> TestDownShardTolerantSearch, DocValuesMultiTest, OverseerStatusTest,
> FieldMutatingUpdateProcessorTest, DistributedFacetPivotLongTailTest,
> CachingDirectoryFactoryTest, TestRemoveLastDynamicCopyField,
> TestSolrQueryParser, TestCSVResponseWriter, TestRestoreCore,
> NoCacheHeaderTest, TestCharFilters, RuleEngineTest,
> TestIndexingPerformance, DistributedExpandComponentTest, RankQueryTest,
> TestReplicationHandlerBackup, HighlighterConfigTest, DebugComponentTest,
> TestIntervalFaceting, TestSearchPerf, TestAnalyzeInfixSuggestions,
> BadIndexSchemaTest, TestReRankQParserPlugin, TestDefaultStatsCache,
> CloneFieldUpdateProcessorFactoryTest,
> TestSolrQueryParserDefaultOperatorResource, LeaderElectionTest,
> HardAutoCommitTest, TestSolrDeletionPolicy1, UnloadDistributedZkTest,
> TestDefaultSearchFieldResource, BasicDistributedZk2Test,
> TestInfoStreamLogging, TestFieldTypeCollectionResource,
> HdfsBasicDistributedZkTest, CircularListTest,
> TestCopyFieldCollectionResource, CurrencyFieldXmlFileTest,
> TestCoreDiscovery, HdfsCollectionsAPIDistributedZkTest,
> TestExceedMaxTermLength, DistribCursorPagingTest, TestCloudManagedSchema,
> FacetPivotSmallTest, SearchHandlerTest, TestSortingResponseWriter,
> TestStandardQParsers, HdfsSyncSliceTest, SuggesterTSTTest,
> LeaderInitiatedRecoveryOnShardRestartTest, TestJmxIntegration,
> TestFuzzyAnalyzedSuggestions, AddBlockUpdateTest, TestReload,
> TermVectorComponentDistributedTest, BasicDistributedZkTest, SyncSliceTest,
> OpenCloseCoreStressTest, OverseerTest]
> [junit4] Completed [253/545] on J2 in 53.73s, 12 tests, 1 failure, 1
> skipped <<< FAILURES!
>
> [...truncated 919 lines...]
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452:
> The following error occurred while executing this line:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006:
> There were test failures: 545 suites (7 ignored), 2158 tests, 1 failure, 51
> ignored (37 assumptions) [seed: B47F6E4FF608FD26]
>
> Total time: 61 minutes 54 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3725 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3725/
1 tests failed.
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([B47F6E4FF608FD26:1F2B897363D25E28]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10417 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_B47F6E4FF608FD26-001/init-core-data-001
[junit4] 2> 960760 INFO (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
[junit4] 2> 960760 INFO (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 960760 INFO (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 960800 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplay
[junit4] 2> 960800 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 960811 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 960811 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 960911 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer start zk server on port:54238
[junit4] 2> 960911 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 960937 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 960952 INFO (zkCallback-535-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7a62b1 name:ZooKeeperConnection Watcher:127.0.0.1:54238/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 960955 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 960955 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 960955 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 960964 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 960969 INFO (zkCallback-536-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2911d84 name:ZooKeeperConnection Watcher:127.0.0.1:54238 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 960969 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 960969 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 960976 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 960982 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961017 INFO (zkCallback-537-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e85766f name:ZooKeeperConnection Watcher:127.0.0.1:54238 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961017 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961017 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961017 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 961020 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 961021 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 961022 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 961023 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 961024 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 961025 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 961027 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 961029 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961042 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961055 INFO (zkCallback-538-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@44977f5d name:ZooKeeperConnection Watcher:127.0.0.1:54238/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961056 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961056 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961056 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 961057 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 961059 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 961060 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 961060 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822931338756099-127.0.0.1:54238_solr-n_0000000000
[junit4] 2> 961061 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54238_solr
[junit4] 2> 961061 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 961062 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931338756099-127.0.0.1:54238_solr-n_0000000000) starting
[junit4] 2> 961063 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 961065 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 961067 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 961068 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 961070 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 961073 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 961089 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 961090 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 961092 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 961092 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 961094 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 961094 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 961105 INFO (OverseerCollectionConfigSetProcessor-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 961115 INFO (zkCallback-538-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961122 INFO (zkCallback-535-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961150 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"} current state version: 2
[junit4] 2> 961150 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 961253 INFO (zkCallback-535-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961253 INFO (zkCallback-538-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961319 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54238 54238
[junit4] 2> 961441 INFO (Thread-1963) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54238 54238
[junit4] 2> 961442 WARN (Thread-1963) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/collections
[junit4] 2> 2 /solr/live_nodes
[junit4] 2>
[junit4] 2> 961442 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReplay
[junit4] 2> 961442 INFO (TEST-OverseerTest.testReplay-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931338756099-127.0.0.1:54238_solr-n_0000000000) closing
[junit4] 2> 961445 INFO (OverseerStateUpdate-94822931338756099-127.0.0.1:54238_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54238_solr
[junit4] 2> 961469 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerFailure
[junit4] 2> 961469 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 961481 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 961488 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 961488 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 961581 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer start zk server on port:44090
[junit4] 2> 961581 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961597 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961613 INFO (zkCallback-540-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7306b0a name:ZooKeeperConnection Watcher:127.0.0.1:44090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961613 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961613 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961621 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961641 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961641 INFO (zkCallback-541-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14c7ad60 name:ZooKeeperConnection Watcher:127.0.0.1:44090 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961641 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961641 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961641 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 961667 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961674 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961675 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a828b25 name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961675 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961675 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961675 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 961677 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 961678 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 961679 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 961680 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 961681 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 961682 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961718 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961745 INFO (zkCallback-543-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70bf6912 name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961745 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961745 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961747 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 961748 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
[junit4] 2> 961749 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961750 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 961767 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 961775 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 961798 INFO (zkCallback-544-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@471a9a7a name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 961798 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 961799 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 961799 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 961800 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 961801 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 961802 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 961803 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822931383648260-127.0.0.1:44090_solr-n_0000000000
[junit4] 2> 961803 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44090_solr
[junit4] 2> 961803 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 961804 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) starting
[junit4] 2> 961805 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 961807 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 961808 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 961810 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 961811 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 961813 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 961816 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 961825 INFO (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 961826 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 961826 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 962843 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
[junit4] 2> 962844 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 962844 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 962844 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 962845 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 962845 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 962845 INFO (zkCallback-544-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 963343 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 963344 INFO (zkCallback-544-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 963345 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 963345 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 963349 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
[junit4] 2> 963350 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 963353 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 963356 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 963357 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "core":"core1"}
[junit4] 2> 963360 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://node1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 963361 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 963363 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
[junit4] 2> 963363 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 963363 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 963366 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000001
[junit4] 2> 963366 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector was going to be leader /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000001 , seq(0) /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
[junit4] 2> 963466 INFO (zkCallback-544-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 963466 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 963468 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 963828 INFO (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:355)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:172)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 963829 INFO (OverseerCollectionConfigSetProcessor-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) am no longer a leader.
[junit4] 2> 964522 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 964524 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000002
[junit4] 2> 964524 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector was going to be leader /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000002 , seq(0) /collections/collection1/leader_elect/shard1/election/94822931383648259-node1_core1-n_0000000000
[junit4] 2> 964525 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 964525 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 964529 INFO (zkCallback-546-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6254332a name:ZooKeeperConnection Watcher:127.0.0.1:44090/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 964529 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 964529 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 964529 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931383648260-127.0.0.1:44090_solr-n_0000000000) closing
[junit4] 2> 964530 INFO (OverseerStateUpdate-94822931383648260-127.0.0.1:44090_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44090_solr
[junit4] 2> 964530 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 964531 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 964532 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 964533 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822931383648261-127.0.0.1:44090_solr-n_0000000001
[junit4] 2> 964533 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44090_solr
[junit4] 2> 964533 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 964535 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931383648261-127.0.0.1:44090_solr-n_0000000001) starting
[junit4] 2> 964542 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 964543 INFO (OverseerCollectionConfigSetProcessor-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 964544 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 964546 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 964553 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"} current state version: 2
[junit4] 2> 964553 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 964554 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 964556 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 964556 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 964556 INFO (zkCallback-546-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 964608 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1"} current state version: 3
[junit4] 2> 964710 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.o.ZkStateWriter going to delete_collection /collections/collection1/state.json
[junit4] 2> 964716 INFO (zkCallback-546-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 964716 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 964716 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 964717 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 964717 INFO (zkCallback-546-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 964717 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 965218 INFO (zkCallback-546-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 965218 INFO (zkCallback-542-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 965218 INFO (zkCallback-543-thread-2) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 965223 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44090 44090
[junit4] 2> 965408 INFO (Thread-1964) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44090 44090
[junit4] 2> 965408 WARN (Thread-1964) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/collections
[junit4] 2> 4 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 965409 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerFailure
[junit4] 2> 965409 INFO (TEST-OverseerTest.testOverseerFailure-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931383648261-127.0.0.1:44090_solr-n_0000000001) closing
[junit4] 2> 965409 INFO (OverseerStateUpdate-94822931383648261-127.0.0.1:44090_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44090_solr
[junit4] 2> 965410 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 965415 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testStateChange
[junit4] 2> 965416 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 965418 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 965418 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 965518 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer start zk server on port:50085
[junit4] 2> 965518 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 965519 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 965527 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e204284 name:ZooKeeperConnection Watcher:127.0.0.1:50085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 965527 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 965527 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 965527 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 965529 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 965533 INFO (zkCallback-549-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bc872ab name:ZooKeeperConnection Watcher:127.0.0.1:50085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 965533 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 965534 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 965534 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x150e0f435010001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 965535 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 965535 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 965553 INFO (zkCallback-550-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2eb503e2 name:ZooKeeperConnection Watcher:127.0.0.1:50085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 965553 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 965553 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 965553 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 965560 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 965561 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 965562 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 965564 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 965565 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 965566 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 965567 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 965577 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 965577 INFO (zkCallback-551-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bfbf3f8 name:ZooKeeperConnection Watcher:127.0.0.1:50085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 965577 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 965578 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 965578 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 965579 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 965580 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 965582 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 965583 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822931640680451-127.0.0.1:50085_solr-n_0000000000
[junit4] 2> 965583 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50085_solr
[junit4] 2> 965583 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 965584 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931640680451-127.0.0.1:50085_solr-n_0000000000) starting
[junit4] 2> 965586 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 965588 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 965590 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 965592 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 965594 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 965596 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 965599 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 965600 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 965600 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 965605 INFO (OverseerCollectionConfigSetProcessor-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 965607 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"} current state version: 0
[junit4] 2> 965608 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 965608 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 965609 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 965609 INFO (zkCallback-551-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 965660 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 965660 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 965660 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator node=core_node1 is already registered
[junit4] 2> 965660 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
[junit4] 2> 965763 INFO (zkCallback-548-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 965763 INFO (zkCallback-551-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 965812 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50085 50085
[junit4] 2> 965969 INFO (Thread-1965) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50085 50085
[junit4] 2> 965970 WARN (Thread-1965) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/collections
[junit4] 2> 2 /solr/live_nodes
[junit4] 2>
[junit4] 2> 965970 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateChange
[junit4] 2> 965970 INFO (TEST-OverseerTest.testStateChange-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931640680451-127.0.0.1:50085_solr-n_0000000000) closing
[junit4] 2> 965970 INFO (OverseerStateUpdate-94822931640680451-127.0.0.1:50085_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:50085_solr
[junit4] 2> 965977 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 965982 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testShardAssignment
[junit4] 2> 965983 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 965983 INFO (Thread-1966) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 965983 INFO (Thread-1966) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 966083 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer start zk server on port:37594
[junit4] 2> 966083 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 966089 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 966094 INFO (zkCallback-553-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3edc922 name:ZooKeeperConnection Watcher:127.0.0.1:37594 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 966096 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 966096 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 966098 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 966101 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 966113 INFO (zkCallback-554-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6cc287e name:ZooKeeperConnection Watcher:127.0.0.1:37594 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 966115 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 966115 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 966115 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 966122 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 966123 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 966137 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29d2100c name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 966137 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 966137 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 966138 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 966140 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 966141 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 966142 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 966145 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 966147 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 966148 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 966151 INFO (zkCallback-556-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@131dfc89 name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 966151 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 966151 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 966152 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 966153 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 966154 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 966155 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 966156 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822931677773827-127.0.0.1:37594_solr-n_0000000000
[junit4] 2> 966156 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37594_solr
[junit4] 2> 966156 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 966159 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822931677773827-127.0.0.1:37594_solr-n_0000000000) starting
[junit4] 2> 966160 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 966162 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 966164 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 966167 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 966170 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 966172 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 966176 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 966177 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 966178 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 966183 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 966184 INFO (OverseerCollectionConfigSetProcessor-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 966187 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 966193 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 966197 INFO (zkCallback-558-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22fcbcf5 name:ZooKeeperConnection Watcher:127.0.0.1:37594/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 966197 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 966197 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 966199 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 966202 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
[junit4] 2> 966203 INFO (zkCallback-556-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 966206 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 966206 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 0
[junit4] 2> 966208 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 966208 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
[junit4] 2> 966208 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 966211 INFO (zkCallback-556-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 966211 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 966214 INFO (zkCallback-558-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4] 2> 966217 INFO (zkCallback-558-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 966706 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 966707 INFO (zkCallback-558-thread-2) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 966707 INFO (zkCallback-556-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 966707 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
[junit4] 2> 966710 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard2/election/94822931677773828-127.0.0.1_core1-n_0000000000
[junit4] 2> 966711 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
[junit4] 2> 966713 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 966715 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core1"} current state version: 1
[junit4] 2> 966717 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 1
[junit4] 2> 966717 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 966717 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 966717 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 966821 INFO (zkCallback-556-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 966821 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 966821 INFO (zkCallback-558-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 967216 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 967220 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94822931677773828-127.0.0.1_core2-n_0000000000
[junit4] 2> 967220 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
[junit4] 2> 967222 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 967224 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core2"} current state version: 2
[junit4] 2> 967226 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 2
[junit4] 2> 967227 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 967227 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 967227 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 967329 INFO (zkCallback-558-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 967329 INFO (zkCallback-556-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 967329 INFO (zkCallback-555-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 967726 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 967730 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard3/election/94822931677773828-127.0.0.1_core3-n_0000000000
[junit4] 2> 967731 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
[junit4] 2> 967733 INFO (TEST-OverseerTest.testShardAssignment-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 967736 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core3"} current state version: 3
[junit4] 2> 967738 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 3
[junit4] 2> 967739 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 967739 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
[junit4] 2> 967739 INFO (OverseerStateUpdate-94822931677773827-127.0.0.1:37594_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 967841 INFO (zkCallback-556-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - up
[...truncated too long message...]
Changed path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009218 INFO (zkCallback-656-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009218 INFO (zkCallback-657-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009614 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard3/election
[junit4] 2> 1009617 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard3/election/94822934316384260-127.0.0.1_core3-n_0000000000
[junit4] 2> 1009618 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard3
[junit4] 2> 1009619 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 1009621 INFO (OverseerStateUpdate-94822934316384259-127.0.0.1:44386_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "base_url":"http://127.0.0.1/solr/",
[junit4] 2> "core":"core3"} current state version: 7
[junit4] 2> 1009724 INFO (zkCallback-656-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009724 INFO (zkCallback-657-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009724 INFO (zkCallback-659-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009773 INFO (zkCallback-657-thread-2) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1009773 INFO (zkCallback-659-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1011273 WARN (zkCallback-659-thread-1) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1011275 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44386 44386
[junit4] 2> 1011433 INFO (Thread-1974) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44386 44386
[junit4] 2> 1011434 WARN (Thread-1974) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/live_nodes
[junit4] 2>
[junit4] 2> 1011434 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
[junit4] 2> 1011434 INFO (TEST-OverseerTest.testBadQueueItem-seed#[B47F6E4FF608FD26]) [ ] o.a.s.c.Overseer Overseer (id=94822934316384259-127.0.0.1:44386_solr-n_0000000000) closing
[junit4] 2> 1011435 INFO (OverseerStateUpdate-94822934316384259-127.0.0.1:44386_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44386_solr
[junit4] 2> 1011436 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
[junit4] 2> 1014435 INFO (SUITE-OverseerTest-seed#[B47F6E4FF608FD26]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_B47F6E4FF608FD26-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=pt_PT, timezone=America/Port-au-Prince
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=287501920,total=506462208
[junit4] 2> NOTE: All tests run in this JVM: [RequestHandlersTest, TestReloadAndDeleteDocs, DateRangeFieldTest, TestWriterPerf, RollingRestartTest, LukeRequestHandlerTest, QueryParsingTest, TestGroupingSearch, PrimitiveFieldTypeTest, TestRestManager, StatsComponentTest, TestPerFieldSimilarity, TestSolrIndexConfig, HttpPartitionTest, TestConfigSetImmutable, BadComponentTest, SampleTest, CoreAdminHandlerTest, TestFileDictionaryLookup, TestStressReorder, AddSchemaFieldsUpdateProcessorFactoryTest, PluginInfoTest, TestCollationField, SolrCoreCheckLockOnStartupTest, XmlUpdateRequestHandlerTest, TestReversedWildcardFilterFactory, TestSchemaVersionResource, TestOverriddenPrefixQueryForCustomFieldType, TestSolrCloudWithKerberosAlt, TestRequestStatusCollectionAPI, QueryResultKeyTest, BinaryUpdateRequestHandlerTest, UpdateParamsTest, TestDownShardTolerantSearch, DocValuesMultiTest, OverseerStatusTest, FieldMutatingUpdateProcessorTest, DistributedFacetPivotLongTailTest, CachingDirectoryFactoryTest, TestRemoveLastDynamicCopyField, TestSolrQueryParser, TestCSVResponseWriter, TestRestoreCore, NoCacheHeaderTest, TestCharFilters, RuleEngineTest, TestIndexingPerformance, DistributedExpandComponentTest, RankQueryTest, TestReplicationHandlerBackup, HighlighterConfigTest, DebugComponentTest, TestIntervalFaceting, TestSearchPerf, TestAnalyzeInfixSuggestions, BadIndexSchemaTest, TestReRankQParserPlugin, TestDefaultStatsCache, CloneFieldUpdateProcessorFactoryTest, TestSolrQueryParserDefaultOperatorResource, LeaderElectionTest, HardAutoCommitTest, TestSolrDeletionPolicy1, UnloadDistributedZkTest, TestDefaultSearchFieldResource, BasicDistributedZk2Test, TestInfoStreamLogging, TestFieldTypeCollectionResource, HdfsBasicDistributedZkTest, CircularListTest, TestCopyFieldCollectionResource, CurrencyFieldXmlFileTest, TestCoreDiscovery, HdfsCollectionsAPIDistributedZkTest, TestExceedMaxTermLength, DistribCursorPagingTest, TestCloudManagedSchema, FacetPivotSmallTest, SearchHandlerTest, TestSortingResponseWriter, TestStandardQParsers, HdfsSyncSliceTest, SuggesterTSTTest, LeaderInitiatedRecoveryOnShardRestartTest, TestJmxIntegration, TestFuzzyAnalyzedSuggestions, AddBlockUpdateTest, TestReload, TermVectorComponentDistributedTest, BasicDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest]
[junit4] Completed [253/545] on J2 in 53.73s, 12 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 919 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 1 failure, 51 ignored (37 assumptions) [seed: B47F6E4FF608FD26]
Total time: 61 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3724 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3724/
3 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:43804/tzj/e/awholynewcollection_0: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /tzj/e/awholynewcollection_0/select. Reason: <pre> {trace=java.lang.NullPointerException at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43804/tzj/e/awholynewcollection_0: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /tzj/e/awholynewcollection_0/select. Reason:
<pre> {trace=java.lang.NullPointerException
at org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([BD1C5AC370A6D6D8:35486519DE5ABB20]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 1) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 2) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=998, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 4) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
1) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
2) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=998, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
4) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([BD1C5AC370A6D6D8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 3) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
3) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([BD1C5AC370A6D6D8]:0)
Build Log:
[...truncated 9787 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/init-core-data-001
[junit4] 2> 176752 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[BD1C5AC370A6D6D8]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
[junit4] 2> 176752 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[BD1C5AC370A6D6D8]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /tzj/e
[junit4] 2> 176761 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 176761 INFO (Thread-243) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 176761 INFO (Thread-243) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 176861 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.ZkTestServer start zk server on port:56531
[junit4] 2> 176861 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 176863 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 176865 INFO (zkCallback-107-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7ed72ded name:ZooKeeperConnection Watcher:127.0.0.1:56531 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 176865 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 176866 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 176866 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 176869 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 176871 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 176887 INFO (zkCallback-108-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7377e860 name:ZooKeeperConnection Watcher:127.0.0.1:56531/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 176887 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 176887 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 176887 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
[junit4] 2> 176891 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
[junit4] 2> 176893 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
[junit4] 2> 176896 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
[junit4] 2> 176898 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 176899 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 176903 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 176903 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
[junit4] 2> 176906 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 176907 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 176909 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 176910 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
[junit4] 2> 176913 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 176913 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
[junit4] 2> 176915 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 176916 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
[junit4] 2> 176918 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 176918 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 176921 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 176921 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 176924 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 176924 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 176927 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 176927 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 176930 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 176930 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
[junit4] 2> 176934 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 176942 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 176943 INFO (zkCallback-109-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ebd16ca name:ZooKeeperConnection Watcher:127.0.0.1:56531 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 176943 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 176943 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 176943 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 176946 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 176951 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 176951 INFO (zkCallback-110-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1fef9b2e name:ZooKeeperConnection Watcher:127.0.0.1:56531/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 176951 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 176952 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 176952 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 176952 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 176955 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 176955 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
[junit4] 2> 176957 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
[junit4] 2> 176958 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
[junit4] 2> 176959 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 176960 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 176961 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 176962 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
[junit4] 2> 176963 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 176964 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
[junit4] 2> 176965 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 176966 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
[junit4] 2> 176967 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 176968 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 176970 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 176970 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 176972 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 176972 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 176974 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 176974 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
[junit4] 2> 176976 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 176976 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
[junit4] 2> 177335 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1
[junit4] 2> 177337 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.e.j.s.Server jetty-9.2.13.v20150730
[junit4] 2> 177339 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@489d1cbe{/tzj/e,null,AVAILABLE}
[junit4] 2> 177341 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.e.j.s.ServerConnector Started ServerConnector@64cc4691{HTTP/1.1}{127.0.0.1:34890}
[junit4] 2> 177341 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.e.j.s.Server Started @180172ms
[junit4] 2> 177341 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostPort=34890, hostContext=/tzj/e, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/tempDir-001/control/data}
[junit4] 2> 177342 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@7b3cb2c6
[junit4] 2> 177342 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/'
[junit4] 2> 177342 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 177342 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 177361 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 177361 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 177363 INFO (zkCallback-111-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5620f476 name:ZooKeeperConnection Watcher:127.0.0.1:56531/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 177363 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 177364 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 177365 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 177365 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/solr.xml
[junit4] 2> 177381 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores
[junit4] 2> 177381 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.CoreContainer New CoreContainer 1411475904
[junit4] 2> 177381 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/]
[junit4] 2> 177381 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/lib
[junit4] 2> 177381 WARN (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/lib).
[junit4] 2> 177391 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 177392 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 177393 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 177393 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 177393 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56531/solr
[junit4] 2> 177393 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 177393 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 177396 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 177398 INFO (zkCallback-113-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7fdc046a name:ZooKeeperConnection Watcher:127.0.0.1:56531 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 177398 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 177399 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 177400 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 177402 INFO (zkCallback-114-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@212630ff name:ZooKeeperConnection Watcher:127.0.0.1:56531/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 177402 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 177403 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 177405 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 177409 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 177411 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 177413 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 177415 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 177417 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 177418 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 177420 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 177421 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34890_tzj%2Fe
[junit4] 2> 177421 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34890_tzj%2Fe
[junit4] 2> 177423 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 177424 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 177426 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177426 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000
[junit4] 2> 177427 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34890_tzj%2Fe
[junit4] 2> 177427 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 177428 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.Overseer Overseer (id=94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) starting
[junit4] 2> 177430 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 177437 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 177437 INFO (OverseerCollectionConfigSetProcessor-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 177438 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 177438 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 177480 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 177480 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 177482 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores
[junit4] 2> 177483 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1, collection=control_collection, config=solrconfig.xml, dataDir=data/, shard=, transient=false, loadOnStartup=true, name=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/, schema=schema.xml, coreNodeName=, configSetProperties=configsetprops.json}
[junit4] 2> 177483 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/
[junit4] 2> 177483 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.CoresLocator Found 1 core definitions
[junit4] 2> 177498 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController publishing state=down
[junit4] 2> 177498 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
[junit4] 2> 177499 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1
[junit4] 2> 177499 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 177501 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "collection":"control_collection",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34890_tzj%2Fe",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:34890/tzj/e",
[junit4] 2> "shard":null,
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1"} current state version: 0
[junit4] 2> 177501 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController look for our core node name
[junit4] 2> 177501 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "collection":"control_collection",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34890_tzj%2Fe",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:34890/tzj/e",
[junit4] 2> "shard":null,
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1"}
[junit4] 2> 177502 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
[junit4] 2> 177502 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 177504 INFO (zkCallback-114-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 178502 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
[junit4] 2> 178502 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
[junit4] 2> 178502 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ZkController Collection zkNode exists
[junit4] 2> 178503 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
[junit4] 2> 178503 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 178503 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/'
[junit4] 2> 178504 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 178504 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 178521 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0
[junit4] 2> 178528 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 178542 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
[junit4] 2> 178571 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
[junit4] 2> 178573 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 178582 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 178835 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 178844 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema default search field in schema is text
[junit4] 2> 178846 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema unique key field: id
[junit4] 2> 178860 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
[junit4] 2> 178863 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
[junit4] 2> 178866 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 178867 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
[junit4] 2> 178867 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
[junit4] 2> 178867 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 178867 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
[junit4] 2> 178868 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
[junit4] 2> 178868 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1
[junit4] 2> 178869 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 178869 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 178869 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 178869 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2cb88b36
[junit4] 2> 178870 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data
[junit4] 2> 178870 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data/index/
[junit4] 2> 178870 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 178870 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data/index
[junit4] 2> 178870 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=34.3818359375, floorSegmentMB=1.5849609375, forceMergeDeletesPctAllowed=19.448364199609273, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 178871 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7a6a0f16 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4904ac23),segFN=segments_1,generation=1}
[junit4] 2> 178871 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 178872 INFO (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data/
[junit4] 2> 178872 WARN (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_BD1C5AC370A6D6D8-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
[junit4] 2> 178878 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 178878 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
[junit4] 2> 178878 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 178878 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 178879 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 178879 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 178879 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 178879 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 178879 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 178880 WARN (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 178882 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 178884 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 178885 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 178886 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 178891 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/get,/admin/logging,/replication,/config,/admin/plugins,/update/csv,/update/json,/schema,/admin/properties,/update,/admin/segments,/dump,/admin/threads,standard,/admin/file,/admin/system,/admin/luke,/update/json/docs,/admin/ping
[junit4] 2> 178892 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 178893 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 178893 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 178894 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
[junit4] 2> 178894 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
[junit4] 2> 178894 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=43.26171875, floorSegmentMB=0.44140625, forceMergeDeletesPctAllowed=12.477018430492583, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 178895 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7a6a0f16 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4904ac23),segFN=segments_1,generation=1}
[junit4] 2> 178895 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 178895 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@23d020e0[collection1] main
[junit4] 2> 178895 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
[junit4] 2> 178896 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 178896 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
[junit4] 2> 178897 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 178898 INFO (searcherExecutor-315-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@23d020e0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 178898 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
[junit4] 2> 178898 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
[junit4] 2> 178899 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
[junit4] 2> 178899 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1517154566254100480
[junit4] 2> 178901 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1517154566254100480
[junit4] 2> 178901 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
[junit4] 2> 178901 INFO (coreLoadExecutor-314-thread-1-processing-n:127.0.0.1:34890_tzj%2Fe) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
[junit4] 2> 178902 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:34890/tzj/e collection:control_collection shard:shard1
[junit4] 2> 178903 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 178907 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94822160250896390-core_node1-n_0000000000
[junit4] 2> 178907 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
[junit4] 2> 178909 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 178909 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 178909 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34890/tzj/e/collection1/
[junit4] 2> 178909 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 178909 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 178910 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34890/tzj/e/collection1/ has no replicas
[junit4] 2> 178910 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34890/tzj/e/collection1/ shard1
[junit4] 2> 178911 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 178914 INFO (coreZkRegister-308-thread-1-processing-c:control_collection n:127.0.0.1:34890_tzj%2Fe x:collection1 s:shard1 r:core_node1) [n:127.0.0.1:34890_tzj%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
[junit4] 2> 178917 INFO (OverseerStateUpdate-94822160250896390-127.0.0.1:34890_tzj%2Fe-n_0000000000) [n:127.0.0.1:34890_tzj%2Fe ] o.a.s.c.
[...truncated too long message...]
dCredentialsProvidersTest, CollectionsAPIDistributedZkTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=BD1C5AC370A6D6D8 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=Asia/Riyadh87 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
[junit4] > 1) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] > 2) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=998, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > 4) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([BD1C5AC370A6D6D8]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=1185, name=zkCallback-141-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=997, name=TEST-CollectionsAPIDistributedZkTest.test-seed#[BD1C5AC370A6D6D8]-SendThread(127.0.0.1:56531), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 3) Thread[id=999, name=zkCallback-141-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=1186, name=zkCallback-141-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([BD1C5AC370A6D6D8]:0)
[junit4] Completed [61/545] on J1 in 83.51s, 1 test, 3 errors <<< FAILURES!
[...truncated 1507 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 2 suite-level errors, 1 error, 1160 ignored (27 assumptions) [seed: BD1C5AC370A6D6D8]
Total time: 53 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3723 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/3723/
3 tests failed.
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Expected that the LIR state would've been down by now expected:<down> but was:<recovering>
Stack Trace:
java.lang.AssertionError: Expected that the LIR state would've been down by now expected:<down> but was:<recovering>
at __randomizedtesting.SeedInfo.seed([B23FADD1B08186:E6250B6DE83278E7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.apache.solr.cloud.ForceLeaderTest.putNonLeadersIntoLIR(ForceLeaderTest.java:386)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup
Error Message:
Failed to create backup
Stack Trace:
java.lang.AssertionError: Failed to create backup
at __randomizedtesting.SeedInfo.seed([B23FADD1B08186:41391FC8F60E72C9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.handler.CheckBackupStatus.fetchStatus(CheckBackupStatus.java:51)
at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([B23FADD1B08186:ABE6D891446A2288]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9864 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/init-core-data-001
[junit4] 2> 237010 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBackupOnCommit
[junit4] 2> 237010 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/collection1
[junit4] 2> 237020 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.e.j.s.Server jetty-9.2.13.v20150730
[junit4] 2> 237024 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c4e5452{/solr,null,AVAILABLE}
[junit4] 2> 237026 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4d883d32{HTTP/1.1}{127.0.0.1:44892}
[junit4] 2> 237026 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.e.j.s.Server Started @240074ms
[junit4] 2> 237026 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44892, solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/collection1/data}
[junit4] 2> 237026 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
[junit4] 2> 237027 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/'
[junit4] 2> 237027 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 237027 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 237044 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/solr.xml
[junit4] 2> 237059 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/.
[junit4] 2> 237059 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer New CoreContainer 71395209
[junit4] 2> 237059 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/]
[junit4] 2> 237059 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/lib
[junit4] 2> 237059 WARN (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/lib).
[junit4] 2> 237068 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 237069 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 237069 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 237069 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 237069 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 237070 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 237070 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/.
[junit4] 2> 237071 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrCore Created CoreDescriptor: {absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/, loadOnStartup=true, config=solrconfig.xml, dataDir=data/, transient=false, configSetProperties=configsetprops.json, name=collection1, schema=schema.xml, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1}
[junit4] 2> 237071 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/
[junit4] 2> 237071 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Found 1 core definitions
[junit4] 2> 237072 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/'
[junit4] 2> 237072 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 237072 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 237073 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
[junit4] 2> 237073 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 237094 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 237108 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
[junit4] 2> 237139 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
[junit4] 2> 237139 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/conf/schema.xml
[junit4] 2> 237143 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 237152 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.s.IndexSchema unique key field: id
[junit4] 2> 237153 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/conf'
[junit4] 2> 237153 INFO (coreLoadExecutor-751-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/
[junit4] 2> 237153 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 237153 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/], dataDir=[null]
[junit4] 2> 237153 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
[junit4] 2> 237154 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data
[junit4] 2> 237154 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index/
[junit4] 2> 237154 WARN (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 237154 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index
[junit4] 2> 237154 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 237155 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7d5d4c3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@78383770),segFN=segments_1,generation=1}
[junit4] 2> 237155 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 237155 INFO (OldIndexDirectoryCleanupThreadForCore-collection1) [ x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/
[junit4] 2> 237157 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 237160 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 237161 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 237163 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 237164 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 237166 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/plugins,/admin/ping,/update/csv,/config,/update,/replication,/schema,/admin/threads,/update/json,/admin/file,/get,/admin/properties,standard,/admin/system,/admin/luke,/admin/logging,/admin/segments,/update/json/docs
[junit4] 2> 237167 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 237167 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
[junit4] 2> 237167 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
[junit4] 2> 237168 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8923411926016864]
[junit4] 2> 237173 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7d5d4c3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@78383770),segFN=segments_1,generation=1}
[junit4] 2> 237173 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 237174 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@426a4773[collection1] main
[junit4] 2> 237175 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/conf
[junit4] 2> 237176 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/conf}
[junit4] 2> 237176 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/conf
[junit4] 2> 237177 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
[junit4] 2> 237177 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema-replication2.xml:schema.xml
[junit4] 2> 237178 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 237178 INFO (coreLoadExecutor-751-thread-1) [ x:collection1] o.a.s.c.CoreContainer registering core: collection1
[junit4] 2> 237181 INFO (searcherExecutor-752-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@426a4773[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 237577 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 237577 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 0
[junit4] 2> 237581 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
[junit4] 2> 237583 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
[junit4] 2> 237586 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
[junit4] 2> 237588 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 0
[junit4] 2> 237591 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
[junit4] 2> 237593 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 0
[junit4] 2> 237595 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 0
[junit4] 2> 237598 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 0
[junit4] 2> 237600 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
[junit4] 2> 237603 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 0
[junit4] 2> 237605 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
[junit4] 2> 237608 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 0
[junit4] 2> 237611 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
[junit4] 2> 237614 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 0
[junit4] 2> 237617 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
[junit4] 2> 237619 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 0
[junit4] 2> 237621 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0
[junit4] 2> 237624 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 0
[junit4] 2> 237626 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
[junit4] 2> 237629 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 0
[junit4] 2> 237632 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
[junit4] 2> 237635 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 0
[junit4] 2> 237638 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
[junit4] 2> 237641 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 0
[junit4] 2> 237644 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
[junit4] 2> 237646 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 0
[junit4] 2> 237649 INFO (qtp1470762225-1391) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
[junit4] 2> 237652 INFO (qtp1470762225-1393) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 0
[junit4] 2> 237663 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 237667 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7d5d4c3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@78383770),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7d5d4c3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@78383770),segFN=segments_2,generation=2}
[junit4] 2> 237667 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 2
[junit4] 2> 237668 INFO (Thread-560) [ x:collection1] o.a.s.h.SnapShooter Creating backup snapshot <not named> at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/
[junit4] 2> 237669 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@396917f5[collection1] main
[junit4] 2> 237669 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.UpdateHandler end_commit_flush
[junit4] 2> 237671 INFO (Thread-560) [ x:collection1] o.a.s.h.SnapShooter Done creating backup snapshot: <not named> at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/
[junit4] 2> 237671 INFO (searcherExecutor-752-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@396917f5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C28)))}
[junit4] 2> 237673 INFO (qtp1470762225-1392) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={commit=true&waitSearcher=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4] 2> 237676 INFO (qtp1470762225-1387) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 238680 INFO (qtp1470762225-1390) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0
[junit4] 2> 239686 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBackupOnCommit
[junit4] 2> 239687 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@4d883d32{HTTP/1.1}{127.0.0.1:0}
[junit4] 2> 239687 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=71395209
[junit4] 2> 239688 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@475eeee
[junit4] 2> 239688 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=28,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4] 2> 239688 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
[junit4] 2> 239688 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 239688 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index;done=false>>]
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data/index
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data;done=false>>]
[junit4] 2> 239689 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-001/./collection1/data
[junit4] 2> 239690 INFO (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[B23FADD1B08186]) [ x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5c4e5452{/solr,null,UNAVAILABLE}
[junit4] 2> 239694 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestBackup
[junit4] 2> 239695 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/collection1
[junit4] 2> 239699 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.e.j.s.Server jetty-9.2.13.v20150730
[junit4] 2> 239700 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@692e1464{/solr,null,AVAILABLE}
[junit4] 2> 239701 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.e.j.s.ServerConnector Started ServerConnector@503ce72e{HTTP/1.1}{127.0.0.1:45236}
[junit4] 2> 239701 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.e.j.s.Server Started @242749ms
[junit4] 2> 239701 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=45236}
[junit4] 2> 239701 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@3827de24
[junit4] 2> 239701 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/'
[junit4] 2> 239702 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 239702 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 239718 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/solr.xml
[junit4] 2> 239732 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/.
[junit4] 2> 239732 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer New CoreContainer 269669922
[junit4] 2> 239732 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/]
[junit4] 2> 239732 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/lib
[junit4] 2> 239732 WARN (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/lib).
[junit4] 2> 240316 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 240317 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 240318 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 240318 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 240318 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 240318 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 240319 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/.
[junit4] 2> 240320 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.SolrCore Created CoreDescriptor: {instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1, dataDir=data/, schema=schema.xml, config=solrconfig.xml, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/, loadOnStartup=true, name=collection1, configSetProperties=configsetprops.json, transient=false}
[junit4] 2> 240320 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/
[junit4] 2> 240320 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoresLocator Found 1 core definitions
[junit4] 2> 240320 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/'
[junit4] 2> 240321 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2
[junit4] 2> 240321 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 240321 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 240321 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 240341 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 240354 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.4.0
[junit4] 2> 240381 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
[junit4] 2> 240382 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.s.IndexSchema Reading Solr Schema from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/conf/schema.xml
[junit4] 2> 240386 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 240394 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.s.IndexSchema unique key field: id
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/conf'
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/], dataDir=[null]
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
[junit4] 2> 240396 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data
[junit4] 2> 240397 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index/
[junit4] 2> 240397 WARN (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 240397 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index
[junit4] 2> 240397 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 240398 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a6c7819 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1740d77b),segFN=segments_1,generation=1}
[junit4] 2> 240398 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 240399 INFO (OldIndexDirectoryCleanupThreadForCore-collection1) [ x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/
[junit4] 2> 240400 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 240403 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 240404 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 240405 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 240406 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 240409 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/admin/plugins,/admin/ping,/update/csv,/config,/update,/replication,/schema,/admin/threads,/update/json,/admin/file,/get,/admin/properties,standard,/admin/system,/admin/luke,/admin/logging,/admin/segments,/update/json/docs
[junit4] 2> 240410 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 240410 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
[junit4] 2> 240410 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
[junit4] 2> 240410 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8923411926016864]
[junit4] 2> 240411 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a6c7819 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1740d77b),segFN=segments_1,generation=1}
[junit4] 2> 240411 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 240411 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@7e80ae14[collection1] main
[junit4] 2> 240411 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/conf
[junit4] 2> 240411 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/conf}
[junit4] 2> 240412 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/conf
[junit4] 2> 240412 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
[junit4] 2> 240412 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema-replication2.xml:schema.xml
[junit4] 2> 240412 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 240412 INFO (coreLoadExecutor-761-thread-1) [ x:collection1] o.a.s.c.CoreContainer registering core: collection1
[junit4] 2> 240412 INFO (searcherExecutor-762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7e80ae14[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 240825 INFO (qtp1181023513-1408) [ x:collection1] o.a.s.c.SolrCore [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 240825 INFO (qtp1181023513-1408) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 0
[junit4] 2> 240829 INFO (qtp1181023513-1412) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
[junit4] 2> 240831 INFO (qtp1181023513-1410) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
[junit4] 2> 240834 INFO (qtp1181023513-1413) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
[junit4] 2> 240836 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 240838 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a6c7819 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1740d77b),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4a6c7819 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1740d77b),segFN=segments_2,generation=2}
[junit4] 2> 240838 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.c.SolrCore newest commit generation = 2
[junit4] 2> 240848 INFO (Thread-575) [ x:collection1] o.a.s.h.SnapShooter Creating backup snapshot <not named> at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/
[junit4] 2> 240850 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@33de2bff[collection1] main
[junit4] 2> 240850 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.u.UpdateHandler end_commit_flush
[junit4] 2> 240850 INFO (searcherExecutor-762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33de2bff[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.4.0):C3)))}
[junit4] 2> 240850 INFO (qtp1181023513-1411) [ x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/solr path=/update params={commit=true&waitSearcher=true&wt=javabin&version=2&softCommit=false} {commit=} 0 14
[junit4] 2> 240857 INFO (qtp1181023513-1408) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=backup} status=0 QTime=3
[junit4] 2> 240858 INFO (Thread-576) [ x:collection1] o.a.s.h.SnapShooter Creating backup snapshot <not named> at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/
[junit4] 2> 240862 INFO (Thread-576) [ x:collection1] o.a.s.h.SnapShooter Done creating backup snapshot: <not named> at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/
[junit4] 2> 240872 ERROR (Thread-575) [ x:collection1] o.a.s.h.SnapShooter Exception while creating snapshot
[junit4] 2> java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/collection1/data/snapshot.20151107062312841/_0_FST50_0.doc
[junit4] 2> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[junit4] 2> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[junit4] 2> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
[junit4] 2> at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
[junit4] 2> at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:430)
[junit4] 2> at java.nio.file.Files.newOutputStream(Files.java:172)
[junit4] 2> at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:276)
[junit4] 2> at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:225)
[junit4] 2> at org.apache.lucene.store.Directory.copyFrom(Directory.java:151)
[junit4] 2> at org.apache.solr.handler.SnapShooter.copyFiles(SnapShooter.java:206)
[junit4] 2> at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:139)
[junit4] 2> at org.apache.solr.handler.SnapShooter$1.run(SnapShooter.java:88)
[junit4] 2> 240881 INFO (qtp1181023513-1412) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=8
[junit4] 2> 240882 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTestBackup
[junit4] 2> 240889 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@503ce72e{HTTP/1.1}{127.0.0.1:0}
[junit4] 2> 240889 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=269669922
[junit4] 2> 240890 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6e4cb318
[junit4] 2> 240890 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.UpdateHandler closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=3,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4] 2> 240890 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
[junit4] 2> 240890 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 240890 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data;done=false>>]
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index;done=false>>]
[junit4] 2> 240891 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001/solr-instance-002/./collection1/data/index
[junit4] 2> 240892 INFO (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[B23FADD1B08186]) [ x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@692e1464{/solr,null,UNAVAILABLE}
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandlerBackup -Dtests.method=doTestBackup -Dtests.seed=B23FADD1B08186 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en_NZ -Dtests.timezone=Asia/Jayapura -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 1.21s J2 | TestReplicationHandlerBackup.doTestBackup <<<
[junit4] > Throwable #1: java.lang.AssertionError: Failed to create backup
[junit4] > at __randomizedtesting.SeedInfo.seed([B23FADD1B08186:41391FC8F60E72C9]:0)
[junit4] > at org.apache.solr.handler.CheckBackupStatus.fetchStatus(CheckBackupStatus.java:51)
[junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:197)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 240898 INFO (SUITE-TestReplicationHandlerBackup-seed#[B23FADD1B08186]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandlerBackup_B23FADD1B08186-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {name=FST50, id=FST50}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=en_NZ, timezone=Asia/Jayapura
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=237819120,total=376438784
[junit4] 2> NOTE: All tests run in this JVM: [FieldMutatingUpdateProcessorTest, TestCollationField, TestElisionMultitermQuery, QueryParsingTest, DistribCursorPagingTest, TestShardHandlerFactory, LeaderElectionTest, BinaryUpdateRequestHandlerTest, RecoveryZkTest, TestJmxMonitoredMap, DistributedExpandComponentTest, CoreAdminCreateDiscoverTest, SolrCoreCheckLockOnStartupTest, DirectUpdateHandlerTest, TestRecovery, SecurityConfHandlerTest, RegexBoostProcessorTest, TestRestoreCore, DocExpirationUpdateProcessorFactoryTest, CachingDirectoryFactoryTest, TermVectorComponentDistributedTest, TestFieldTypeResource, TestInfoStreamLogging, TestGroupingSearch, TestAddFieldRealTimeGet, TestJettySolrRunner, TestHashQParserPlugin, TestPostingsSolrHighlighter, TestFieldTypeCollectionResource, TestUtils, TestSuggestSpellingConverter, BJQParserTest, TestSweetSpotSimilarityFactory, TestReRankQParserPlugin, TestObjectReleaseTracker, TestReplicationHandlerBackup]
[junit4] Completed [69/545] on J2 in 3.93s, 2 tests, 1 failure <<< FAILURES!
[...truncated 595 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.OverseerTest_B23FADD1B08186-001/init-core-data-001
[junit4] 2> 1021404 INFO (SUITE-OverseerTest-seed#[B23FADD1B08186]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
[junit4] 2> 1021404 INFO (SUITE-OverseerTest-seed#[B23FADD1B08186]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1021404 INFO (SUITE-OverseerTest-seed#[B23FADD1B08186]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 1021409 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplay
[junit4] 2> 1021410 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1021410 INFO (Thread-4448) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1021410 INFO (Thread-4448) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1021510 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.ZkTestServer start zk server on port:53178
[junit4] 2> 1021510 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1021511 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021513 INFO (zkCallback-953-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@46c47813 name:ZooKeeperConnection Watcher:127.0.0.1:53178/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1021513 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021514 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1021514 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1021514 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021516 INFO (zkCallback-954-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b9828d4 name:ZooKeeperConnection Watcher:127.0.0.1:53178 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1021516 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021516 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1021517 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1021518 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021519 INFO (zkCallback-955-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@189e92f0 name:ZooKeeperConnection Watcher:127.0.0.1:53178 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1021519 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021519 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1021519 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 1021523 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 1021525 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 1021526 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 1021528 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 1021530 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 1021531 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 1021535 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 1021537 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1021538 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1021539 INFO (zkCallback-956-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a4736a5 name:ZooKeeperConnection Watcher:127.0.0.1:53178/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1021539 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1021540 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1021540 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
[junit4] 2> 1021542 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 1021543 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 1021545 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1021545 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94820484629921795-127.0.0.1:53178_solr-n_0000000000
[junit4] 2> 1021545 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53178_solr
[junit4] 2> 1021546 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 1021548 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.Overseer Overseer (id=94820484629921795-127.0.0.1:53178_solr-n_0000000000) starting
[junit4] 2> 1021549 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 1021551 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 1021554 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 1021556 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 1021558 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 1021563 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1021564 INFO (OverseerCollectionConfigSetProcessor-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 1021564 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 1021565 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 1021570 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: workQueueSize: 2, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1021571 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1021571 INFO (zkCallback-953-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021571 INFO (zkCallback-956-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021573 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: workQueueSize: 0, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1021578 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1021580 INFO (zkCallback-956-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021580 INFO (zkCallback-953-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021582 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"} current state version: 2
[junit4] 2> 1021583 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1021686 INFO (zkCallback-953-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021686 INFO (zkCallback-956-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1021768 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53178 53178
[junit4] 2> 1021945 INFO (Thread-4448) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53178 53178
[junit4] 2> 1021946 WARN (Thread-4448) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 1021946 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReplay
[junit4] 2> 1021946 INFO (TEST-OverseerTest.testReplay-seed#[B23FADD1B08186]) [ ] o.a.s.c.Overseer Overseer (id=94820484629921795-127.0.0.1:53178_solr-n_0000000000) closing
[junit4] 2> 1021947 INFO (OverseerStateUpdate-94820484629921795-127.0.0.1:53178_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:53178_solr
[junit4] 2> 1021952 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.ch
[...truncated too long message...]
unit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2> 1520260 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@a367c7d{HTTP/1.1}{127.0.0.1:0}
[junit4] 2> 1520261 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@781e2d88{/,null,UNAVAILABLE}
[junit4] 2> 1520262 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44894 44894
[junit4] 2> 1520518 INFO (Thread-3819) [n:127.0.0.1:35617_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44894 44894
[junit4] 2> 1520520 WARN (Thread-3819) [n:127.0.0.1:35617_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 2 /solr/overseer_elect/election/94820513510719492-127.0.0.1:48164_-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 1520520 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:45594/, target: http://127.0.0.1:43977/
[junit4] 2> 1520521 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:57566/, target: http://127.0.0.1:40545/
[junit4] 2> 1520521 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:41649/, target: http://127.0.0.1:50710/
[junit4] 2> 1520521 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[B23FADD1B08186]) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SocketProxy Closing 2 connections to: http://127.0.0.1:48164/, target: http://127.0.0.1:46753/
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ForceLeaderTest -Dtests.method=testReplicasInLIRNoLeader -Dtests.seed=B23FADD1B08186 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es_HN -Dtests.timezone=Brazil/Acre -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 58.2s J0 | ForceLeaderTest.testReplicasInLIRNoLeader <<<
[junit4] > Throwable #1: java.lang.AssertionError: Expected that the LIR state would've been down by now expected:<down> but was:<recovering>
[junit4] > at __randomizedtesting.SeedInfo.seed([B23FADD1B08186:E6250B6DE83278E7]:0)
[junit4] > at org.apache.solr.cloud.ForceLeaderTest.putNonLeadersIntoLIR(ForceLeaderTest.java:386)
[junit4] > at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:92)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1520524 INFO (SUITE-ForceLeaderTest-seed#[B23FADD1B08186]-worker) [n:127.0.0.1:45594_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_B23FADD1B08186-001
[junit4] 2> Nov 06, 2015 9:44:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {intDefault=PostingsFormat(name=LuceneFixedGap), range_facet_l=PostingsFormat(name=MockRandom), text=BlockTreeOrds(blocksize=128), timestamp=PostingsFormat(name=MockRandom), a_t=PostingsFormat(name=MockRandom), _version_=PostingsFormat(name=LuceneFixedGap), range_facet_l_dv=FSTOrd50, multiDefault=PostingsFormat(name=MockRandom), range_facet_i_dv=PostingsFormat(name=MockRandom), id=FSTOrd50}, docValues:{timestamp=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=es_HN, timezone=Brazil/Acre
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=41090960,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [TestReplicaProperties, TestDocSet, FullHLLTest, CollectionTooManyReplicasTest, TestScoreJoinQPNoScore, TestCloudPivotFacet, TestCollectionAPI, PolyFieldTest, HdfsWriteToMultipleCollectionsTest, DistributedQueryElevationComponentTest, TestAuthorizationFramework, TestStressUserVersions, JavabinLoaderTest, JSONWriterTest, LeaderFailoverAfterPartitionTest, TestConfig, TestExactSharedStatsCache, AnalysisAfterCoreReloadTest, TestLRUCache, CloudMLTQParserTest, LeaderInitiatedRecoveryOnCommitTest, BasicZkTest, DistributedFacetPivotLargeTest, LeaderElectionIntegrationTest, SharedFSAutoReplicaFailoverTest, TestDynamicLoading, TestEmbeddedSolrServerConstructors, ParsingFieldUpdateProcessorsTest, BigEndianAscendingWordDeserializerTest, TestAnalyzedSuggestions, TestCollationFieldDocValues, TestConfigSetsAPIZkFailure, TestFaceting, DistributedSuggestComponentTest, ZkNodePropsTest, ActionThrottleTest, SegmentsInfoRequestHandlerTest, TestCloudInspectUtil, CopyFieldTest, TestConfigSetsAPIExclusivity, TestTrackingShardHandlerFactory, TestCryptoKeys, TestOrdValues, RemoteQueryErrorTest, TestReplicationHandler, StandardRequestHandlerTest, ReturnFieldsTest, AlternateDirectoryTest, TestSolrConfigHandlerCloud, TestOmitPositions, FileBasedSpellCheckerTest, EchoParamsTest, SharedFSAutoReplicaFailoverUtilsTest, DistributedDebugComponentTest, HdfsBasicDistributedZk2Test, VMParamsZkACLAndCredentialsProvidersTest, TestRealTimeGet, TestPhraseSuggestions, TestFunctionQuery, CurrencyFieldOpenExchangeTest, SuggestComponentTest, TestDistributedSearch, TestLuceneMatchVersion, DisMaxRequestHandlerTest, ConcurrentDeleteAndCreateCollectionTest, PeerSyncTest, ResourceLoaderTest, RAMDirectoryFactoryTest, MultiThreadedOCPTest, TestSolr4Spatial, TestRangeQuery, SignatureUpdateProcessorFactoryTest, TestManagedSchema, HLLSerializationTest, CacheHeaderTest, TestStressLucene, TestJoin, TestPseudoReturnFields, ResponseLogComponentTest, TestSolrXml, TestMinMaxOnMultiValuedField, TestChildDocTransformer, TestNRTOpen, TestLeaderElectionZkExpiry, TestLMDirichletSimilarityFactory, HdfsChaosMonkeyNothingIsSafeTest, TestJsonFacets, DefaultValueUpdateProcessorTest, SolrRequestParserTest, SolrIndexSplitterTest, SimplePostToolTest, TestExtendedDismaxParser, TestCSVLoader, TestUpdate, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, DocValuesMultiTest, DebugComponentTest, TestSurroundQueryParser, TestWriterPerf, TestQueryTypes, RequestHandlersTest, TermVectorComponentTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, TestJmxIntegration, TestComponentsName, TestQuerySenderListener, TestQuerySenderNoQuery, TestStressRecovery, ScriptEngineTest, PluginInfoTest, PreAnalyzedFieldTest, DOMUtilTest, SliceStateTest, UUIDFieldTest, TestRTGBase, DistributedIntervalFacetingTest, CollectionReloadTest, CustomCollectionTest, ForceLeaderTest]
[junit4] Completed [409/545] on J0 in 133.75s, 3 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 441 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2158 tests, 3 failures, 51 ignored (37 assumptions) [seed: B23FADD1B08186]
Total time: 61 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3722 - Failure
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Looking into them. Possibly broken due to SOLR-7989.
On Fri, Nov 6, 2015 at 10:55 PM, Chris Hostetter <ho...@fucit.org>
wrote:
>
> These recent jenkins OverseerTest faiulres (seems to have started failing
> yesterday for the first time) all have seeds that reproduce 100% reliably
> for me...
>
>
> 5x...
>
> ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
> -Dtests.seed=7B352AEEA690CBAB -Dtests.multiplier=2 -Dtests.slow=true
> -Dtests.locale=hu -Dtests.timezone=NET -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>
> ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
> -Dtests.seed=DF592108C7E1C290 -Dtests.slow=true -Dtests.locale=sv_SE
> -Dtests.timezone=Africa/Maputo -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>
>
> Trunk...
>
> ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
> -Dtests.seed=7771E2C402B621D6 -Dtests.slow=true -Dtests.locale=en_AU
> -Dtests.timezone=Brazil/DeNoronha -Dtests.asserts=true
> -Dtests.file.encoding=Cp1252
>
> ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
> -Dtests.seed=B3178102A7C387E9 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=cy_GB -Dtests.timezone=ECT -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1
>
>
>
>
>
>
> :
> : 1 tests failed.
> : FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
> :
> : Error Message:
> : expected:<1> but was:<2>
> :
> : Stack Trace:
> : java.lang.AssertionError: expected:<1> but was:<2>
> : at
> __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
> : at org.junit.Assert.fail(Assert.java:93)
> : at org.junit.Assert.failNotEquals(Assert.java:647)
> : at org.junit.Assert.assertEquals(Assert.java:128)
> : at org.junit.Assert.assertEquals(Assert.java:472)
> : at org.junit.Assert.assertEquals(Assert.java:456)
> : at
> org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
> : at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> : at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> : at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> : at java.lang.reflect.Method.invoke(Method.java:606)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
> : at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> : at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> : at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> : at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> : at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> : at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> : at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
> : at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
> : at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> : at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> : at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> : at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
> : at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> : at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> : at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> : at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> : at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> : at java.lang.Thread.run(Thread.java:745)
> :
> :
> :
> :
> : Build Log:
> : [...truncated 9839 lines...]
> : [junit4] Suite: org.apache.solr.cloud.OverseerTest
> : [junit4] 2> Creating dataDir:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001/init-core-data-001
> : [junit4] 2> 48955 INFO
> (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
> : [junit4] 2> 48956 INFO
> (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ####initCore
> : [junit4] 2> 48956 INFO
> (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ####initCore end
> : [junit4] 2> 48962 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testDoubleAssignment
> : [junit4] 2> 48963 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> : [junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> : [junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer
> Starting server
> : [junit4] 2> 49063 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:51889
> : [junit4] 2> 49063 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49064 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49068 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@68781cb5
> name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49068 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49069 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49069 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49070 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49072 INFO (zkCallback-49-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@4a44db2a
> name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49073 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49073 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49076 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49077 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49079 INFO (zkCallback-50-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6759c069
> name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49079 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49079 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49079 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> : [junit4] 2> 49085 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> : [junit4] 2> 49088 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> : [junit4] 2> 49090 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> : [junit4] 2> 49092 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> : [junit4] 2> 49094 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> : [junit4] 2> 49095 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 49099 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49099 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49101 INFO (zkCallback-51-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@76ac9512
> name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49101 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49102 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49105 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 49108 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
> : [junit4] 2> 49109 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 49109 INFO (zkCallback-51-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 49110 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49111 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49113 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@190452f0
> name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49113 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49113 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49114 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> : [junit4] 2> 49121 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> : [junit4] 2> 49123 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> : [junit4] 2> 49125 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> : [junit4] 2> 49126 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94818801242931204-127.0.0.1:51889_solr-n_0000000000
> : [junit4] 2> 49126 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51889
> _solr
> : [junit4] 2> 49126 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> : [junit4] 2> 49128 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) starting
> : [junit4] 2> 49130 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> : [junit4] 2> 49133 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> : [junit4] 2> 49136 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> : [junit4] 2> 49139 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> : [junit4] 2> 49141 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> : [junit4] 2> 49144 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> : [junit4] 2> 49150 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> : [junit4] 2> 49150 INFO
> (OverseerCollectionConfigSetProcessor-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> : [junit4] 2> 49151 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> : [junit4] 2> 49152 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 49157 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 0
> : [junit4] 2> 49158 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"}
> : [junit4] 2> 49158 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
> : [junit4] 2> 49158 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> : [junit4] 2> 49161 INFO (zkCallback-51-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49161 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49161 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49654 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> : [junit4] 2> 49656 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 49656 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 49656 INFO (zkCallback-51-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 49661 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94818801242931203-node1_core1-n_0000000000
> : [junit4] 2> 49662 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
> : [junit4] 2> 49665 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 49668 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 49669 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"}
> : [junit4] 2> 49673 INFO (zkCallback-51-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 49673 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 49678 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 49678 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 49679 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 49680 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 49685 INFO (zkCallback-54-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7c4f31de
> name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 49686 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 49686 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 49695 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 49698 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
> : [junit4] 2> 49700 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 49700 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 49700 INFO (zkCallback-54-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 49703 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> : [junit4] 2> 49704 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 1
> : [junit4] 2> 49705 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"}
> : [junit4] 2> 49705 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
> : [junit4] 2> 49708 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94818801242931205-node1_core1-n_0000000001
> : [junit4] 2> 49710 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 49714 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 49715 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"}
> : [junit4] 2> 49722 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 49831 INFO (zkCallback-52-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49831 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49831 INFO (zkCallback-54-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 49836 INFO (zkCallback-54-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 49836 INFO (zkCallback-48-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 49845 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:51889 51889
> : [junit4] 2> 50045 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:51889 51889
> : [junit4] 2> 50046 WARN (Thread-85) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> : [junit4] 2> Maximum concurrent create/delete watches above limit:
> : [junit4] 2>
> : [junit4] 2> 4 /solr/aliases.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent data watches above limit:
> : [junit4] 2>
> : [junit4] 2> 4 /solr/clusterstate.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent children watches above limit:
> : [junit4] 2>
> : [junit4] 2> 4 /solr/collections
> : [junit4] 2> 4 /solr/live_nodes
> : [junit4] 2>
> : [junit4] 2> 50047 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testDoubleAssignment
> : [junit4] 2> 50047 INFO
> (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) closing
> : [junit4] 2> 50047 INFO
> (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51889_solr
> : [junit4] 2> 50057 ERROR (OverseerExitThread) [ ]
> o.a.s.c.Overseer could not read the data
> : [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> : [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> : [junit4] 2> 50060 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
> : [junit4] 2> 50061 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> : [junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> : [junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer
> Starting server
> : [junit4] 2> 50161 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:56334
> : [junit4] 2> 50161 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 50162 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 50165 INFO (zkCallback-55-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@4a4111b2
> name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 50165 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 50165 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 50168 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 50168 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 50170 INFO (zkCallback-56-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@51182c3c
> name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 50170 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 50170 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 50170 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> : [junit4] 2> 50174 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 50174 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 50180 INFO (zkCallback-57-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@4f355f55
> name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 50180 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 50181 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 50181 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> : [junit4] 2> 50183 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> : [junit4] 2> 50185 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> : [junit4] 2> 50187 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> : [junit4] 2> 50189 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> : [junit4] 2> 50191 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 50193 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 50194 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 50196 INFO (zkCallback-58-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@5336a302
> name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 50196 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 50197 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 50200 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 50203 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
> : [junit4] 2> 50205 INFO (zkCallback-58-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 50205 INFO (zkCallback-57-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 50205 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> : [junit4] 2> 50208 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> : [junit4] 2> 50210 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> : [junit4] 2> 50211 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> : [junit4] 2> 50212 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94818801314693122-127.0.0.1:56334_solr-n_0000000000
> : [junit4] 2> 50213 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56334
> _solr
> : [junit4] 2> 50213 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> : [junit4] 2> 50215 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) starting
> : [junit4] 2> 50217 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> : [junit4] 2> 50220 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> : [junit4] 2> 50224 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> : [junit4] 2> 50227 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> : [junit4] 2> 50230 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> : [junit4] 2> 50233 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> : [junit4] 2> 50238 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> : [junit4] 2> 50238 INFO
> (OverseerCollectionConfigSetProcessor-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> : [junit4] 2> 50240 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> : [junit4] 2> 50244 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"} current state
> version: 0
> : [junit4] 2> 50244 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"recovering",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://node1/solr/"}
> : [junit4] 2> 50244 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
> : [junit4] 2> 50244 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> : [junit4] 2> 50246 INFO (zkCallback-58-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 50246 INFO (zkCallback-57-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 50741 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> : [junit4] 2> 50743 INFO (zkCallback-57-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 50743 INFO (zkCallback-58-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 50749 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94818801314693123-node1_core1-n_0000000000
> : [junit4] 2> 50753 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
> : [junit4] 2> 50757 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 50761 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 50761 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "core_node_name":"core_node1",
> : [junit4] 2> "core":"core1"}
> : [junit4] 2> 50764 INFO (zkCallback-57-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 50764 INFO (zkCallback-58-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 50764 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://node1/solr/",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 50785 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:56334 56334
> : [junit4] 2> 50887 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:56334 56334
> : [junit4] 2> 50889 WARN (Thread-86) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> : [junit4] 2> Maximum concurrent create/delete watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/aliases.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent data watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/clusterstate.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent children watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/collections
> : [junit4] 2> 2 /solr/live_nodes
> : [junit4] 2>
> : [junit4] 2> 50889 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
> : [junit4] 2> 50889 INFO
> (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) closing
> : [junit4] 2> 50890 INFO
> (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56334_solr
> : [junit4] 2> 50897 ERROR (OverseerExitThread) [ ]
> o.a.s.c.Overseer could not read the data
> : [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> : [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> : [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest
> -Dtests.method=testOverseerStatsReset -Dtests.seed=7B352AEEA690CBAB
> -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hu
> -Dtests.timezone=NET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> : [junit4] FAILURE 0.85s J2 | OverseerTest.testOverseerStatsReset <<<
> : [junit4] > Throwable #1: java.lang.AssertionError: expected:<1>
> but was:<2>
> : [junit4] > at
> __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
> : [junit4] > at
> org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
> : [junit4] > at java.lang.Thread.run(Thread.java:745)
> : [junit4] 2> 50910 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem
> : [junit4] 2> 50911 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> : [junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> : [junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer
> Starting server
> : [junit4] 2> 51011 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:60811
> : [junit4] 2> 51011 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 51012 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 51015 INFO (zkCallback-60-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@3f19b154
> name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 51015 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 51016 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 51018 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 51018 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 51020 INFO (zkCallback-61-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@22e9d784
> name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 51020 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 51021 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 51021 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> : [junit4] 2> 51024 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 51024 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 51026 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6f0d6162
> name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 51026 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 51027 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 51027 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> : [junit4] 2> 51029 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> : [junit4] 2> 51035 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> : [junit4] 2> 51037 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> : [junit4] 2> 51039 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> : [junit4] 2> 51041 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 51041 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 51043 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@7d0bfa0d
> name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 51043 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 51044 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 51044 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with
> params: socketTimeout=600000&connTimeout=60000&retry=true
> : [junit4] 2> 51046 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect
> : [junit4] 2> 51048 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
> : [junit4] 2> 51050 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer (id=null) closing
> : [junit4] 2> 51051 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /overseer_elect/election/94818801370398723-127.0.0.1:60811_solr-n_0000000000
> : [junit4] 2> 51051 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60811
> _solr
> : [junit4] 2> 51052 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
> : [junit4] 2> 51053 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) starting
> : [junit4] 2> 51055 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> : [junit4] 2> 51058 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> : [junit4] 2> 51060 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> : [junit4] 2> 51063 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> : [junit4] 2> 51065 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> : [junit4] 2> 51068 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> : [junit4] 2> 51072 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> : [junit4] 2> 51073 INFO
> (OverseerCollectionConfigSetProcessor-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> : [junit4] 2> 51073 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 51074 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> : [junit4] 2> 51075 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 51076 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 51077 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 51081 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@286bb11a
> name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 51081 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 51081 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 51083 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 51086 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
> : [junit4] 2> 51090 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 51090 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 51091 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 0)
> : [junit4] 2> 51095 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 0
> : [junit4] 2> 51095 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 51096 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
> : [junit4] 2> 51096 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
> : [junit4] 2> 51098 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 51098 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 51098 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 51594 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard3/election
> : [junit4] 2> 51596 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 51596 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 51597 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 51603 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
> : [junit4] 2> 51604 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
> : [junit4] 2> 51611 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 51615 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard3",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core1"} current state version: 1
> : [junit4] 2> 51618 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core2",
> : [junit4] 2> "core_node_name":"node2",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 1
> : [junit4] 2> 51618 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core2",
> : [junit4] 2> "core_node_name":"node2",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 51619 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> : [junit4] 2> 51619 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
> : [junit4] 2> 51722 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 51722 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 51723 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52117 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard2/election
> : [junit4] 2> 52122 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
> : [junit4] 2> 52123 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
> : [junit4] 2> 52126 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 52130 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard2",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core2"} current state version: 2
> : [junit4] 2> 52133 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core3",
> : [junit4] 2> "core_node_name":"node3",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 2
> : [junit4] 2> 52134 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core3",
> : [junit4] 2> "core_node_name":"node3",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 52134 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> : [junit4] 2> 52134 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> : [junit4] 2> 52238 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52238 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52238 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52631 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection1/leader_elect/shard1/election
> : [junit4] 2> 52638 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection1/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
> : [junit4] 2> 52639 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
> : [junit4] 2> 52642 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 52646 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core3"} current state version: 3
> : [junit4] 2> 52750 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52750 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52750 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52799 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core0",
> : [junit4] 2> "core_node_name":"node0",
> : [junit4] 2> "collection":"",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 4
> : [junit4] 2> 52800 ERROR
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message
> because it has no collection specified: {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core0",
> : [junit4] 2> "core_node_name":"node0",
> : [junit4] 2> "collection":"",
> : [junit4] 2> "numShards":"1",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 52802 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"deletecore",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core0",
> : [junit4] 2> "core_node_name":"node0",
> : [junit4] 2> "collection":""} current state version: 4
> : [junit4] 2> 52802 ERROR
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message
> because it has no collection specified: {
> : [junit4] 2> "operation":"deletecore",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core0",
> : [junit4] 2> "core_node_name":"node0",
> : [junit4] 2> "collection":""}
> : [junit4] 2> 52804 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"node1",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 4
> : [junit4] 2> 52805 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "core_node_name":"node1",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 52805 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection2
> : [junit4] 2> 52805 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
> : [junit4] 2> 52909 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52909 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 52909 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53302 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection2/leader_elect/shard3/election
> : [junit4] 2> 53304 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 53304 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 53305 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A collections change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/collections, has
> occurred - updating...
> : [junit4] 2> 53309 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection2/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
> : [junit4] 2> 53310 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard3
> : [junit4] 2> 53313 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 53317 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard3",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core1"} current state version: 5
> : [junit4] 2> 53320 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core2",
> : [junit4] 2> "core_node_name":"node2",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 5
> : [junit4] 2> 53321 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core2",
> : [junit4] 2> "core_node_name":"node2",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 53321 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> : [junit4] 2> 53321 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
> : [junit4] 2> 53426 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53426 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53426 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53819 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection2/leader_elect/shard2/election
> : [junit4] 2> 53826 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection2/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
> : [junit4] 2> 53827 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard2
> : [junit4] 2> 53830 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 53833 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard2",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core2"} current state version: 6
> : [junit4] 2> 53836 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core3",
> : [junit4] 2> "core_node_name":"node3",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state
> version: 6
> : [junit4] 2> 53837 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "state":"active",
> : [junit4] 2> "node_name":"127.0.0.1",
> : [junit4] 2> "core":"core3",
> : [junit4] 2> "core_node_name":"node3",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "numShards":"3",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
> : [junit4] 2> 53837 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
> : [junit4] 2> 53837 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> : [junit4] 2> 53941 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53941 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 53941 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 54335 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath:
> /collections/collection2/leader_elect/shard1/election
> : [junit4] 2> 54345 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.LeaderElector Joined leadership election with path:
> /collections/collection2/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
> : [junit4] 2> 54347 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard1
> : [junit4] 2> 54350 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
> : [junit4] 2> 54355 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"leader",
> : [junit4] 2> "shard":"shard1",
> : [junit4] 2> "collection":"collection2",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr/",
> : [junit4] 2> "core":"core3"} current state version: 7
> : [junit4] 2> 54460 INFO (zkCallback-62-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 54460 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 54460 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 1)
> : [junit4] 2> 54506 INFO (zkCallback-63-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 54507 INFO (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A live node change: WatchedEvent
> state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred
> - updating... (live nodes size: 1)
> : [junit4] 2> 56007 WARN (zkCallback-65-thread-1) [ ]
> o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
> ZK
> : [junit4] 2> 56010 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:60811 60811
> : [junit4] 2> 56253 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:60811 60811
> : [junit4] 2> 56256 WARN (Thread-87) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> : [junit4] 2> Maximum concurrent create/delete watches above limit:
> : [junit4] 2>
> : [junit4] 2> 3 /solr/aliases.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent data watches above limit:
> : [junit4] 2>
> : [junit4] 2> 3 /solr/clusterstate.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent children watches above limit:
> : [junit4] 2>
> : [junit4] 2> 3 /solr/collections
> : [junit4] 2> 3 /solr/live_nodes
> : [junit4] 2>
> : [junit4] 2> 56256 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
> : [junit4] 2> 56256 INFO
> (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) closing
> : [junit4] 2> 56256 INFO
> (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60811_solr
> : [junit4] 2> 56259 ERROR (OverseerExitThread) [ ]
> o.a.s.c.Overseer could not read the data
> : [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> : [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> : [junit4] 2> 56267 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testReplay
> : [junit4] 2> 56268 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> : [junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> : [junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer
> Starting server
> : [junit4] 2> 56368 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer start zk server on port:43766
> : [junit4] 2> 56368 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 56371 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 56376 INFO (zkCallback-66-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@de91d1d
> name:ZooKeeperConnection Watcher:127.0.0.1:43766/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 56376 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 56376 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 56377 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 56377 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 56379 INFO (zkCallback-67-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@20a01f5b
> name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 56379 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 56380 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 56381 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
> : [junit4] 2> 56382 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
> : [junit4] 2> 56383 INFO (zkCallback-68-thread-1) [ ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@6318cd44
> name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
> : [junit4] 2> 56383 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
> : [junit4] 2> 56384 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient Using default ZkACLProvider
> : [junit4] 2> 56384 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /solr
> : [junit4] 2> 56387 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /live_nodes
> : [junit4] 2> 56389 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /collections
> : [junit4] 2> 56391 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /aliases.json
> : [junit4] 2> 56393 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
> : [junit4] 2> 56395 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /security.json
> : [junit4] 2> 56397 INFO
> (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 56401 INFO (TEST-OverseerTest.testReplay-seed
> :
> : [...truncated too long message...]
> :
> : nit4] 2> 105808 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue
> : [junit4] 2> 105810 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
> : [junit4] 2> 105813 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
> : [junit4] 2> 105815 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
> : [junit4] 2> 105817 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
> : [junit4] 2> 105819 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
> : [junit4] 2> 105822 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.OverseerAutoReplicaFailoverThread Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=30000
> autoReplicaFailoverBadNodeExpiration=60000
> : [junit4] 2> 105829 INFO
> (OverseerCollectionConfigSetProcessor-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer
> operations
> : [junit4] 2> 105833 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Starting to work on the main queue
> : [junit4] 2> 105834 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
> : [junit4] 2> 105837 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "roles":"",
> : [junit4] 2> "state":"recovering"} current state version: 0
> : [junit4] 2> 105837 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "roles":"",
> : [junit4] 2> "state":"recovering"}
> : [junit4] 2> 105838 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
> : [junit4] 2> 105840 INFO (zkCallback-157-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> : [junit4] 2> 105840 INFO (zkCallback-154-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> : [junit4] 2> 105896 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "roles":"",
> : [junit4] 2> "state":"active"} current state version: 1
> : [junit4] 2> 105897 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
> : [junit4] 2> "operation":"state",
> : [junit4] 2> "base_url":"http://127.0.0.1/solr",
> : [junit4] 2> "node_name":"node1",
> : [junit4] 2> "collection":"collection1",
> : [junit4] 2> "core":"core1",
> : [junit4] 2> "roles":"",
> : [junit4] 2> "state":"active"}
> : [junit4] 2> 105897 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator node=core_node1 is already registered
> : [junit4] 2> 105897 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
> : [junit4] 2> 106000 INFO (zkCallback-154-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> : [junit4] 2> 106000 INFO (zkCallback-157-thread-1) [ ]
> o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
> state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has
> occurred - updating... (live nodes size: 0)
> : [junit4] 2> 106053 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.ZkTestServer connecting to 127.0.0.1:40794 40794
> : [junit4] 2> 106225 INFO (Thread-96) [ ] o.a.s.c.ZkTestServer
> connecting to 127.0.0.1:40794 40794
> : [junit4] 2> 106226 WARN (Thread-96) [ ] o.a.s.c.ZkTestServer
> Watch limit violations:
> : [junit4] 2> Maximum concurrent create/delete watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/aliases.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent data watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/clusterstate.json
> : [junit4] 2>
> : [junit4] 2> Maximum concurrent children watches above limit:
> : [junit4] 2>
> : [junit4] 2> 2 /solr/collections
> : [junit4] 2> 2 /solr/live_nodes
> : [junit4] 2>
> : [junit4] 2> 106226 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testStateChange
> : [junit4] 2> 106226 INFO
> (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ]
> o.a.s.c.Overseer Overseer
> (id=94818804959215619-127.0.0.1:40794_solr-n_0000000000) closing
> : [junit4] 2> 106227 INFO
> (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000)
> [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40794_solr
> : [junit4] 2> 106228 ERROR (OverseerExitThread) [ ]
> o.a.s.c.Overseer could not read the data
> : [junit4] 2>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leader
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> : [junit4] 2> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> : [junit4] 2> at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> : [junit4] 2> at
> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
> : [junit4] 2> at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
> : [junit4] 2> 109228 INFO
> (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ]
> o.a.s.SolrTestCaseJ4 ###deleteCore
> : [junit4] 2> NOTE: leaving temporary files on disk at:
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001
> : [junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {},
> docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {},
> locale=hu, timezone=NET
> : [junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
> 1.7.0_80 (64-bit)/cpus=4,threads=1,free=198038552,total=343932928
> : [junit4] 2> NOTE: All tests run in this JVM: [HardAutoCommitTest,
> TestPerFieldSimilarity, FieldMutatingUpdateProcessorTest,
> VMParamsZkACLAndCredentialsProvidersTest, VersionInfoTest,
> LukeRequestHandlerTest, DeleteLastCustomShardedReplicaTest,
> ActionThrottleTest, OverseerTest]
> : [junit4] Completed [24/545] on J2 in 60.31s, 12 tests, 1 failure, 1
> skipped <<< FAILURES!
> :
> : [...truncated 1651 lines...]
> : BUILD FAILED
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452:
> The following error occurred while executing this line:
> :
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006:
> There were test failures: 545 suites (7 ignored), 2156 tests, 1 failure, 51
> ignored (37 assumptions) [seed: 7B352AEEA690CBAB]
> :
> : Total time: 60 minutes 54 seconds
> : Build step 'Invoke Ant' marked build as failure
> : Archiving artifacts
> : Recording test results
> : Email was triggered for: Failure - Any
> : Sending email for trigger: Failure - Any
> :
> :
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: [JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 3722 - Failure
Posted by Chris Hostetter <ho...@fucit.org>.
These recent jenkins OverseerTest faiulres (seems to have started failing
yesterday for the first time) all have seeds that reproduce 100% reliably
for me...
5x...
ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
-Dtests.seed=7B352AEEA690CBAB -Dtests.multiplier=2 -Dtests.slow=true
-Dtests.locale=hu -Dtests.timezone=NET -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
-Dtests.seed=DF592108C7E1C290 -Dtests.slow=true -Dtests.locale=sv_SE
-Dtests.timezone=Africa/Maputo -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
Trunk...
ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
-Dtests.seed=7771E2C402B621D6 -Dtests.slow=true -Dtests.locale=en_AU
-Dtests.timezone=Brazil/DeNoronha -Dtests.asserts=true
-Dtests.file.encoding=Cp1252
ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset
-Dtests.seed=B3178102A7C387E9 -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=cy_GB -Dtests.timezone=ECT -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
:
: 1 tests failed.
: FAILED: org.apache.solr.cloud.OverseerTest.testOverseerStatsReset
:
: Error Message:
: expected:<1> but was:<2>
:
: Stack Trace:
: java.lang.AssertionError: expected:<1> but was:<2>
: at __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
: at org.junit.Assert.fail(Assert.java:93)
: at org.junit.Assert.failNotEquals(Assert.java:647)
: at org.junit.Assert.assertEquals(Assert.java:128)
: at org.junit.Assert.assertEquals(Assert.java:472)
: at org.junit.Assert.assertEquals(Assert.java:456)
: at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: at java.lang.reflect.Method.invoke(Method.java:606)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
: at java.lang.Thread.run(Thread.java:745)
:
:
:
:
: Build Log:
: [...truncated 9839 lines...]
: [junit4] Suite: org.apache.solr.cloud.OverseerTest
: [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001/init-core-data-001
: [junit4] 2> 48955 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
: [junit4] 2> 48956 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
: [junit4] 2> 48956 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
: [junit4] 2> 48962 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDoubleAssignment
: [junit4] 2> 48963 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
: [junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 48963 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer Starting server
: [junit4] 2> 49063 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:51889
: [junit4] 2> 49063 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49064 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49068 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68781cb5 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49068 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49069 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49069 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49070 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49072 INFO (zkCallback-49-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a44db2a name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49073 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49073 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49076 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49077 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49079 INFO (zkCallback-50-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6759c069 name:ZooKeeperConnection Watcher:127.0.0.1:51889 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49079 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
: [junit4] 2> 49085 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
: [junit4] 2> 49088 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
: [junit4] 2> 49090 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
: [junit4] 2> 49092 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
: [junit4] 2> 49094 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
: [junit4] 2> 49095 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 49099 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49099 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49101 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@76ac9512 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49101 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49102 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49105 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 49108 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
: [junit4] 2> 49109 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 49109 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 49110 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49111 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49113 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@190452f0 name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49113 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49113 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49114 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
: [junit4] 2> 49121 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
: [junit4] 2> 49123 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
: [junit4] 2> 49125 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
: [junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801242931204-127.0.0.1:51889_solr-n_0000000000
: [junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51889_solr
: [junit4] 2> 49126 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
: [junit4] 2> 49128 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) starting
: [junit4] 2> 49130 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
: [junit4] 2> 49133 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
: [junit4] 2> 49136 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
: [junit4] 2> 49139 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
: [junit4] 2> 49141 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
: [junit4] 2> 49144 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
: [junit4] 2> 49150 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 49150 INFO (OverseerCollectionConfigSetProcessor-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
: [junit4] 2> 49151 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
: [junit4] 2> 49152 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 49157 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
: [junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
: [junit4] 2> 49158 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
: [junit4] 2> 49161 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49161 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49161 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49654 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 49656 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 49656 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 49656 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 49661 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801242931203-node1_core1-n_0000000000
: [junit4] 2> 49662 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 49665 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 49668 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 49669 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"}
: [junit4] 2> 49673 INFO (zkCallback-51-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49673 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49678 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49678 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 49679 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 49680 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 49685 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c4f31de name:ZooKeeperConnection Watcher:127.0.0.1:51889/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 49686 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 49686 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 49695 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 49698 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
: [junit4] 2> 49700 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 49700 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 49700 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 49703 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 49704 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"} current state version: 1
: [junit4] 2> 49705 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 49705 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
: [junit4] 2> 49708 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801242931205-node1_core1-n_0000000001
: [junit4] 2> 49710 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 49714 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 49715 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"}
: [junit4] 2> 49722 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 49831 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49831 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49831 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49836 INFO (zkCallback-54-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49836 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 49845 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51889 51889
: [junit4] 2> 50045 INFO (Thread-85) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51889 51889
: [junit4] 2> 50046 WARN (Thread-85) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 4 /solr/aliases.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 4 /solr/clusterstate.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 4 /solr/collections
: [junit4] 2> 4 /solr/live_nodes
: [junit4] 2>
: [junit4] 2> 50047 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDoubleAssignment
: [junit4] 2> 50047 INFO (TEST-OverseerTest.testDoubleAssignment-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801242931204-127.0.0.1:51889_solr-n_0000000000) closing
: [junit4] 2> 50047 INFO (OverseerStateUpdate-94818801242931204-127.0.0.1:51889_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51889_solr
: [junit4] 2> 50057 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
: [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
: [junit4] 2> 50060 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
: [junit4] 2> 50061 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
: [junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 50061 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer Starting server
: [junit4] 2> 50161 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:56334
: [junit4] 2> 50161 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 50162 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 50165 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a4111b2 name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 50165 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 50165 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 50168 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 50168 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 50170 INFO (zkCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@51182c3c name:ZooKeeperConnection Watcher:127.0.0.1:56334 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 50170 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
: [junit4] 2> 50174 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 50174 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 50180 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f355f55 name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 50180 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 50181 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 50181 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
: [junit4] 2> 50183 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
: [junit4] 2> 50185 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
: [junit4] 2> 50187 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
: [junit4] 2> 50189 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
: [junit4] 2> 50191 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 50193 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 50194 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 50196 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5336a302 name:ZooKeeperConnection Watcher:127.0.0.1:56334/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 50196 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 50197 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 50200 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 50203 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
: [junit4] 2> 50205 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 50205 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 50205 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
: [junit4] 2> 50208 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
: [junit4] 2> 50210 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
: [junit4] 2> 50211 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
: [junit4] 2> 50212 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801314693122-127.0.0.1:56334_solr-n_0000000000
: [junit4] 2> 50213 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56334_solr
: [junit4] 2> 50213 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
: [junit4] 2> 50215 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) starting
: [junit4] 2> 50217 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
: [junit4] 2> 50220 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
: [junit4] 2> 50224 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
: [junit4] 2> 50227 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
: [junit4] 2> 50230 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
: [junit4] 2> 50233 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
: [junit4] 2> 50238 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 50238 INFO (OverseerCollectionConfigSetProcessor-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
: [junit4] 2> 50240 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
: [junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"} current state version: 0
: [junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
: [junit4] 2> 50244 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
: [junit4] 2> 50246 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 50246 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 50741 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 50743 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 50743 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 50749 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801314693123-node1_core1-n_0000000000
: [junit4] 2> 50753 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 50757 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 50761 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 50761 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "core":"core1"}
: [junit4] 2> 50764 INFO (zkCallback-57-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 50764 INFO (zkCallback-58-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 50764 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://node1/solr/",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 50785 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56334 56334
: [junit4] 2> 50887 INFO (Thread-86) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56334 56334
: [junit4] 2> 50889 WARN (Thread-86) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/aliases.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/clusterstate.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/collections
: [junit4] 2> 2 /solr/live_nodes
: [junit4] 2>
: [junit4] 2> 50889 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
: [junit4] 2> 50889 INFO (TEST-OverseerTest.testOverseerStatsReset-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801314693122-127.0.0.1:56334_solr-n_0000000000) closing
: [junit4] 2> 50890 INFO (OverseerStateUpdate-94818801314693122-127.0.0.1:56334_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56334_solr
: [junit4] 2> 50897 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
: [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
: [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerStatsReset -Dtests.seed=7B352AEEA690CBAB -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hu -Dtests.timezone=NET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
: [junit4] FAILURE 0.85s J2 | OverseerTest.testOverseerStatsReset <<<
: [junit4] > Throwable #1: java.lang.AssertionError: expected:<1> but was:<2>
: [junit4] > at __randomizedtesting.SeedInfo.seed([7B352AEEA690CBAB:D061CDD2334A68A5]:0)
: [junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerStatsReset(OverseerTest.java:722)
: [junit4] > at java.lang.Thread.run(Thread.java:745)
: [junit4] 2> 50910 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem
: [junit4] 2> 50911 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
: [junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 50911 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer Starting server
: [junit4] 2> 51011 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:60811
: [junit4] 2> 51011 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 51012 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 51015 INFO (zkCallback-60-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f19b154 name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 51015 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 51016 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 51018 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 51018 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 51020 INFO (zkCallback-61-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22e9d784 name:ZooKeeperConnection Watcher:127.0.0.1:60811 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 51020 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 51021 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 51021 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
: [junit4] 2> 51024 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 51024 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 51026 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f0d6162 name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 51026 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 51027 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 51027 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
: [junit4] 2> 51029 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
: [junit4] 2> 51035 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
: [junit4] 2> 51037 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
: [junit4] 2> 51039 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
: [junit4] 2> 51041 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 51041 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 51043 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d0bfa0d name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 51043 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 51044 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 51044 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
: [junit4] 2> 51046 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
: [junit4] 2> 51048 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
: [junit4] 2> 51050 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=null) closing
: [junit4] 2> 51051 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94818801370398723-127.0.0.1:60811_solr-n_0000000000
: [junit4] 2> 51051 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60811_solr
: [junit4] 2> 51052 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
: [junit4] 2> 51053 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) starting
: [junit4] 2> 51055 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
: [junit4] 2> 51058 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
: [junit4] 2> 51060 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
: [junit4] 2> 51063 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
: [junit4] 2> 51065 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
: [junit4] 2> 51068 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
: [junit4] 2> 51072 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 51073 INFO (OverseerCollectionConfigSetProcessor-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
: [junit4] 2> 51073 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 51074 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
: [junit4] 2> 51075 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 51076 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 51077 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 51081 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@286bb11a name:ZooKeeperConnection Watcher:127.0.0.1:60811/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 51081 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 51081 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 51083 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 51086 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
: [junit4] 2> 51090 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 51090 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 51091 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 51095 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 0
: [junit4] 2> 51095 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 51096 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
: [junit4] 2> 51096 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
: [junit4] 2> 51098 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 51098 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 51098 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 51594 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard3/election
: [junit4] 2> 51596 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 51596 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 51597 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 51603 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
: [junit4] 2> 51604 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
: [junit4] 2> 51611 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 51615 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard3",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core1"} current state version: 1
: [junit4] 2> 51618 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "core_node_name":"node2",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 1
: [junit4] 2> 51618 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "core_node_name":"node2",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 51619 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
: [junit4] 2> 51619 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
: [junit4] 2> 51722 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 51722 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 51723 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52117 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard2/election
: [junit4] 2> 52122 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
: [junit4] 2> 52123 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
: [junit4] 2> 52126 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 52130 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard2",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core2"} current state version: 2
: [junit4] 2> 52133 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "core_node_name":"node3",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 2
: [junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "core_node_name":"node3",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
: [junit4] 2> 52134 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
: [junit4] 2> 52238 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52238 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52238 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52631 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 52638 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection1/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
: [junit4] 2> 52639 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 52642 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 52646 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core3"} current state version: 3
: [junit4] 2> 52750 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52750 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52750 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52799 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core0",
: [junit4] 2> "core_node_name":"node0",
: [junit4] 2> "collection":"",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 4
: [junit4] 2> 52800 ERROR (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core0",
: [junit4] 2> "core_node_name":"node0",
: [junit4] 2> "collection":"",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 52802 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"deletecore",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core0",
: [junit4] 2> "core_node_name":"node0",
: [junit4] 2> "collection":""} current state version: 4
: [junit4] 2> 52802 ERROR (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has no collection specified: {
: [junit4] 2> "operation":"deletecore",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core0",
: [junit4] 2> "core_node_name":"node0",
: [junit4] 2> "collection":""}
: [junit4] 2> 52804 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 4
: [junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection2
: [junit4] 2> 52805 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
: [junit4] 2> 52909 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52909 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 52909 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53302 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard3/election
: [junit4] 2> 53304 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 53304 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 53305 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A collections change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections, has occurred - updating...
: [junit4] 2> 53309 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard3/election/94818801370398724-127.0.0.1_core1-n_0000000000
: [junit4] 2> 53310 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard3
: [junit4] 2> 53313 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 53317 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard3",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core1"} current state version: 5
: [junit4] 2> 53320 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "core_node_name":"node2",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 5
: [junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "core_node_name":"node2",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
: [junit4] 2> 53321 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
: [junit4] 2> 53426 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53426 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53426 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53819 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard2/election
: [junit4] 2> 53826 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard2/election/94818801370398724-127.0.0.1_core2-n_0000000000
: [junit4] 2> 53827 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard2
: [junit4] 2> 53830 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 53833 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard2",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core2"} current state version: 6
: [junit4] 2> 53836 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "core_node_name":"node3",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"} current state version: 6
: [junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "core_node_name":"node3",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
: [junit4] 2> 53837 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
: [junit4] 2> 53941 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53941 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 53941 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 54335 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leader_elect/shard1/election
: [junit4] 2> 54345 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.LeaderElector Joined leadership election with path: /collections/collection2/leader_elect/shard1/election/94818801370398724-127.0.0.1_core3-n_0000000000
: [junit4] 2> 54347 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard1
: [junit4] 2> 54350 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
: [junit4] 2> 54355 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"leader",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "collection":"collection2",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/",
: [junit4] 2> "core":"core3"} current state version: 7
: [junit4] 2> 54460 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 54460 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 54460 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 54506 INFO (zkCallback-63-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 54507 INFO (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 56007 WARN (zkCallback-65-thread-1) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK
: [junit4] 2> 56010 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60811 60811
: [junit4] 2> 56253 INFO (Thread-87) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60811 60811
: [junit4] 2> 56256 WARN (Thread-87) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 3 /solr/aliases.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 3 /solr/clusterstate.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 3 /solr/collections
: [junit4] 2> 3 /solr/live_nodes
: [junit4] 2>
: [junit4] 2> 56256 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadQueueItem
: [junit4] 2> 56256 INFO (TEST-OverseerTest.testBadQueueItem-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818801370398723-127.0.0.1:60811_solr-n_0000000000) closing
: [junit4] 2> 56256 INFO (OverseerStateUpdate-94818801370398723-127.0.0.1:60811_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60811_solr
: [junit4] 2> 56259 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
: [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
: [junit4] 2> 56267 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReplay
: [junit4] 2> 56268 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
: [junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 56268 INFO (Thread-88) [ ] o.a.s.c.ZkTestServer Starting server
: [junit4] 2> 56368 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer start zk server on port:43766
: [junit4] 2> 56368 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 56371 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 56376 INFO (zkCallback-66-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@de91d1d name:ZooKeeperConnection Watcher:127.0.0.1:43766/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 56376 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 56376 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 56377 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 56377 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 56379 INFO (zkCallback-67-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@20a01f5b name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 56379 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 56380 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 56381 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
: [junit4] 2> 56382 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
: [junit4] 2> 56383 INFO (zkCallback-68-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6318cd44 name:ZooKeeperConnection Watcher:127.0.0.1:43766 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 56383 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
: [junit4] 2> 56384 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
: [junit4] 2> 56384 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr
: [junit4] 2> 56387 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
: [junit4] 2> 56389 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections
: [junit4] 2> 56391 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
: [junit4] 2> 56393 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
: [junit4] 2> 56395 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /security.json
: [junit4] 2> 56397 INFO (TEST-OverseerTest.testReplay-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 56401 INFO (TEST-OverseerTest.testReplay-seed
:
: [...truncated too long message...]
:
: nit4] 2> 105808 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
: [junit4] 2> 105810 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
: [junit4] 2> 105813 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
: [junit4] 2> 105815 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
: [junit4] 2> 105817 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
: [junit4] 2> 105819 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
: [junit4] 2> 105822 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 105829 INFO (OverseerCollectionConfigSetProcessor-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
: [junit4] 2> 105833 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
: [junit4] 2> 105834 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
: [junit4] 2> 105837 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"} current state version: 0
: [junit4] 2> 105837 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"}
: [junit4] 2> 105838 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
: [junit4] 2> 105840 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 105840 INFO (zkCallback-154-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 105896 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"active"} current state version: 1
: [junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"active"}
: [junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator node=core_node1 is already registered
: [junit4] 2> 105897 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
: [junit4] 2> 106000 INFO (zkCallback-154-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 106000 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 106053 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40794 40794
: [junit4] 2> 106225 INFO (Thread-96) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40794 40794
: [junit4] 2> 106226 WARN (Thread-96) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/aliases.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/clusterstate.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 2 /solr/collections
: [junit4] 2> 2 /solr/live_nodes
: [junit4] 2>
: [junit4] 2> 106226 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateChange
: [junit4] 2> 106226 INFO (TEST-OverseerTest.testStateChange-seed#[7B352AEEA690CBAB]) [ ] o.a.s.c.Overseer Overseer (id=94818804959215619-127.0.0.1:40794_solr-n_0000000000) closing
: [junit4] 2> 106227 INFO (OverseerStateUpdate-94818804959215619-127.0.0.1:40794_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40794_solr
: [junit4] 2> 106228 ERROR (OverseerExitThread) [ ] o.a.s.c.Overseer could not read the data
: [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:87)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:265)
: [junit4] 2> 109228 INFO (SUITE-OverseerTest-seed#[7B352AEEA690CBAB]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
: [junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_7B352AEEA690CBAB-001
: [junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=hu, timezone=NET
: [junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.7.0_80 (64-bit)/cpus=4,threads=1,free=198038552,total=343932928
: [junit4] 2> NOTE: All tests run in this JVM: [HardAutoCommitTest, TestPerFieldSimilarity, FieldMutatingUpdateProcessorTest, VMParamsZkACLAndCredentialsProvidersTest, VersionInfoTest, LukeRequestHandlerTest, DeleteLastCustomShardedReplicaTest, ActionThrottleTest, OverseerTest]
: [junit4] Completed [24/545] on J2 in 60.31s, 12 tests, 1 failure, 1 skipped <<< FAILURES!
:
: [...truncated 1651 lines...]
: BUILD FAILED
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:785: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:729: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:59: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:233: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:526: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1452: The following error occurred while executing this line:
: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1006: There were test failures: 545 suites (7 ignored), 2156 tests, 1 failure, 51 ignored (37 assumptions) [seed: 7B352AEEA690CBAB]
:
: Total time: 60 minutes 54 seconds
: Build step 'Invoke Ant' marked build as failure
: Archiving artifacts
: Recording test results
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
:
:
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org